Monday, January 4, 2010

understanding the android media framework

android media framework is built on top of a set of media libraries, including OpenCORE, vorbis and sonivox. So one of goal of android media framework is to provide a consistent interface for all services provided by underlying libraries and make them transparent to users.

The figure below shows the dependency relationships between libraries of the media framework.

android_media_lib_dep

In this figure, green components are media libraries, yellow components are android internal libraries, grey components are external libraries, and the light blue class is the java consumer of the media framework. Except for android.media.MediaPlayer class, all components are implemented in c or c++.

The core of the media framework is composed of libmedia, libmediaplayerservice and libmedia_jni. Their codes reside in frameworks/base/media folder.

libmedia defines the inheritance hierarchy and base interfaces. It’s the base library.

libmedia_jni is the shim between java application and native library. First, it implements the JNI specification so that it can be used by java application. Second, it implements the facade pattern for the convenience of caller.

libmediaplayerservice implements some of concrete players and the media service which will manage player instances.


The figure below shows the class hierarchy.android_media_classes

This is a simplified version of the class hierarchy. Only some core classes are included. Classes in blue are defined in libmedia, classes in green are defined in libmediaplayerservice, classes in light yellow are defined in binder, which implements the IPC on android. And classes in grey are defined in other libs.

Note the BpInterface and BnInterface are template classes. Any instantiation of them also inherit the template argument INTERFACE as well.

In the class hierarchy diagram, though listed as a separate module, binder is actually implemented inside libutils component whose source code locate at /frameworks/base/libs/utils folder.


An interesting thing to note is in android, the application that intends to show the media content and the player that actually renders the media content run in different process. The red line in the sequence diagram below shows the boundary of two processes.

android_media_sequence

The figure shows three most common operations, creating a new player, setting datasource and playing. The last MediaPlayerBase object is the interface that MediaPlayerService::Client object uses to refer to the concrete player instance. The concrete player can be VorbisPlayer, PVPlayer, or any other player, depending on the type of the media to be played.

When an application creates a android.media.MediaPlayer object, it’s actually holding a proxy which can be used to manipulate the concrete player resides in the mediaserver process. During the whole procedure, two process communicates with Binder IPC mechanism.


Having knowledge above, it’s not difficult to understand why MediaPlayer doesn’t provide an API to use memory stream as source. Because the memory manipulated by the stream is in the address space of the application, and it’s not directly accessible by the mediaserver process.

References:

Google I/O, Mastering the Android Media Framework

android media framework uml diagram

79 comments:

Anonymous said...

thanks,
but pictures does not appear

Unknown said...

thanks for your good post
i have a question if you can help me
i want to know which library does the class Canvas.cpp belong to?
does it belong to the same library (media framework)?
thanks for your help

Unknown said...

to Anonymous
Can you see pictures on other posts?
I can view them on different browsers.

Unknown said...

to rehab
Canvas belongs to the libandroid_runtime lib. It's part of graphic framework.
I think this topic deserves a separate post. :)

Unknown said...

thanks alot for your quick reply
i just want to know the library because i made some changes in the Canvas class but can not update the phone with this changes
thanks again for your help
Rehab

Anonymous said...

i saw it now, it was browser problem
great post, thanks again

Rehab said...

Hi rxwen
i have another question

Android's structure:
Applications.
Application Framework.
Libraries.
Android Runtime.
Linux Kernel.
is this library(libandroid_runtime)
belongs to Libraries or Android Runtime ?

Thanks in advance,
Rehab

Unknown said...

to rehab
It corresponds to the Android runtime in this diagram:
http://developer.android.com/guide/basics/what-is-android.html

Unknown said...

is the core library in the runtime written in java?
because it is in runtime, and in the diagram it has blue color
but the Canvas.cpp is a cpp class
do you have any reason for that?
thanks
Rehab

Unknown said...

Not all of them are implemented in java.
The blue block in the diagrm includes a set of core libs that comprises the foundation of java core library.

Unknown said...

Thanks, i tried it and it worked :)
Rehab

Anonymous said...

nice. found the details that i was looking for. long live rxwen!

André Barbosa said...

can you give more information about stream as source?
if you show the structure for the reception and reproduction of the stream, i appreciate.

Regards.

Unknown said...

dreamer,

I'm so busy with windows ce platform recently. So I'd like to write more about android streaming a bit latter.

André Barbosa said...

ok, good work rxwen.

Regards.

Anonymous said...

Great diagram and valuable information! Could it be possible to wider the diagram to cover also the linux kernel part?

Unknown said...

Google has changed the media framework a lot. I'd like to create a new post to cover new media framework.

Raghu said...

Hi,
How difficult is it to compile the media player and the codecs in the android media framework as a separate media player.

Unknown said...

Can you be more specific what you want to achieve?
The mediaplayer itself is a standalone player that consumes the media framework

Anonymous said...

Hi,

Your article is very helpful. when are your posting new one based on recent framework? eager to have a look

Thanks

sangram said...

Hi rxwen,
Thanks for sharing such a valuable info.
I had one doubt.
Is it possible to integrate any external media player in the media framework like VLC player? If yes, how that can be done?

Thanks,
Sangram

Android app development said...

This is one of the forward post.I like your blog explanation.Thanks for your support.This is one of the special post.
Android app developers

Android developers said...

I have to i appreciate you for the efforts you get in writing this article. so you to promote your blog by using some internet marketing strategy, because it can easily to reach the peoples

yoyo said...

Can we have ActivityManager and WindowManager Class Hierarachy

Anonymous said...

Time for an update:

"...libmedia, libmediaplayerservice and libmedia_jni. Their codes reside in frameworks/base/media folder."

This has been moved to frameworks/av. See here for more details.

Robert Newman said...

Great list for learning about an Android Media Framework. I appreciate you for this informative blog.

Click Devolve for any Mobile App Development needs or suggestions.

Jeevitha S said...

Great explanation and images looks good and easily understandable. I expect this kind of articles in future too.

Jeevitha from Way2Smile (Ruputed App Development Company in Chennai).

Anonymous said...
This comment has been removed by the author.
Evan Raymonds said...

The Sequence Diagram is a good diagram to use to document a system's requirements and to flush out a system's design. The reason the sequence diagram is so useful is because it shows the interaction logic between the objects in the system in the time order that the interactions take place. Want to learn how to draw? Find these sequence diagram examples and templates in the diagram community of Creately.

Unknown said...

Thank you a bunch for sharing this with all of us you actually realize what you are talking about! Bookmarked. Please also seek advice from my site =). We could have a hyperlink change contract between us!

Best Software Development Agency Dubai UAE

Unknown said...

I’ve been surfing online more than three hours today, yet I never found any interesting article like yours. It’s pretty worth enough for me. In my opinion, if all webmasters and bloggers made good content as you did, the web will be a lot more useful than ever before.


Digital marketing Agency

Unknown said...


Nice to be visiting your blog once more, it has been months for me. Well this article that ive been waited for therefore long. i want this article to finish my assignment within the faculty, and it has same topic together with your article. Thanks, nice share.

Web Development Agency Dubai UAE

David Stangley said...

Do you want to learn Quran with tajweed online? We are offering you online Quran tuition at your convenience.

David Stangley said...

Great post, and great website. Thanks for the information!

online female Quran teacher

David Stangley said...

Excellent post. I was always checking this blog, and I’m impressed!

Pakistani clothing websites

David Stangley said...

Your content is nothing short of brilliant in many ways.I think this is engaging and eye-opening material.Thank you so much for caring about your content and your readers.

online Quran teaching academy

David Stangley said...

Good post but I was wondering if you could write a little more on this subject? I’d be very thankful if you could elaborate a little bit further. Appreciate it!

online Quran tutors in UK

David Stangley said...

Nice post mate, keep up the great work, just shared this with my friends.

Quran lessons online


Husnain said...

Our top smm panel provides powerful promoting services to enable you to defeat your competitors. Our Top SMM PANEL tries to provide quality services.

David Carter said...

You composed this post cautiously which is beneficial for us. I got some different information from your article and I will suggest reading this article who needs this info. Thanks for sharing it. Best SMM Panel
SMM Reseller Panel

Numbers Pro said...

Really, I enjoy your site with effective and useful information. It is included very nice post with a lot of our resources. Thanks for share. I enjoy this post. Trusted Tax Accountants in Tarneit
BAS Agent Services in Australia

Unknown said...

Bravo! I admire how you've quoted every sentence with valuable examples and statistics. I must mention your blog has a good readability and less jargon. Appsquadz.com is one such website that posts blogs in the same niche with loaded information.

gotosmmpanel.com said...

Smm Panel | GOTOSMMPANEL

gotosmmpanel said...

https://images.google.vu/url?q=https://gotosmmpanel.com/services

gotosmmpanel said...

http://yelp-pages.com/dispaly-business.php?id=6598&title=

gotosmmpanel.com said...

https://www.hackathon.io/users/244712

gotosmmpanel.com said...

https://gotosmmpanels.mystrikingly.com/

gotosmmpanel.com said...

https://gotosmmpanelzip.blogspot.com/2021/12/smm-panel-with-1-services_52.html

GOTOSMMPANEL.COM said...

https://www.bonanza.com/users/51242703/profile?preview=true

gotosmmpanel said...

https://www.evernote.com/shard/s444/sh/e25378f3-ad3d-16cb-61f1-36042378dfb1/c0b047eec3be8204c1cae74d229aa6ea

GOTOSMMPANEL.COM said...

https://www.evernote.com/shard/s730/sh/e3c40d97-0e06-0680-cd85-a54f15d16ffc/d051afc2b47648f7c258c3a95c72793a

gotosmmpanel said...

https://companylistingnyc.com/listings/smm-panel-with-1-services-gotosmmpanel-com-15/

gotosmmpanel said...

https://site-6385496-9674-2451.mystrikingly.com/blog/smm-panel-with-1-services-gotosmmpanel-com-23b37e78-4271-4780-958a-164dc1a6332d

Ronnie said...

There has been communicate for lots months that hobby fees will be approximately to boom, following file low fees for a few years.follower kaufen

PERYOURHEALTH said...

I was looking for tiktok account for sale. Can anybody suggest to me a good site where I can sale my old tiktok account? I was looking for building my career with tiktok. So that's why I need a good site for this.

gotosmmpanel said...

https://companylistingnyc.com/listings/smm-panel-with-1-services-gotosmmpanel-com-3/

gotosmmpanel said...

https://gotosmmpanel.tumblr.com/post/669559721896853504/get-smm-panel-with-1-top-smm-services

GOTOSMMPANEL.COM said...

https://www.evernote.com/shard/s730/sh/4c6acd5c-19cf-9c5e-06b6-703a1c5a99f7/04a25f3d39e5e4434f6319ec33db7c1c

gotosmmpanel said...

http://gotosmmpanelowner.xtgem.com/__xt_blog/__xtblog_entry/24382953-what-is-the-position-of-the-best-smm-panel#xt_blog

gotosmmpanel said...

https://www.pinterest.com/pin/1034350239388894406

gotosmmpanel said...

http://www.4mark.net/story/5107977/verified-smm-reseller-panel-in-pakistan-gotosmmpanel

Gowripriya said...

Thanks for providing a piece of great information and looking beautiful blog, really nice required information.


CRM Development Companies in Chennai
Top CRM Development Companies in Chennai
CRM Developers in Chennai
software development company in chennai
Top mobile app development companies in Chennai
app developers in chennai

seodxbapps said...

Thanks for sharing such an informative post. In every blog comment, it is essential to post the relevant content. I gained some useful knowledge of things from this article. Anticipating your future posts.

Rising Quran Online School said...

If you’re looking for Online Quran Classes, then you’re in the right place. Rising Quran is an online school, mainly for students from all over the world. We welcome people from all over the world to learn about the Quran and the commandments of Islam regardless of race, color or cast.
Contact us
Learn Online Quran Courses
Learn Online Tuition Courses

Citywebpuneind said...
This comment has been removed by the author.
Citywebpuneind said...

I like the valuable information you provided in your article. I am sure I will learn many new things here! Good luck.
SEO Company Pune
SEO Pune

Alfa Designs said...

This is genuinely an awesome read for me. I have bookmarked it and I am anticipating perusing new articles. Keep doing awesome!
Venetian Mirror
Wine Glass
Bathroom Mirror

Call Center Software said...

You put really very helpful information. Keep it up Call center software | Best call center software

yash said...

Hey, Thanks for asking.

Did you know that there are limited number of global biomedical researchers and scientists to biospecimen suppliers across the globe.

We are one of them trying to reach out the peoples who needs Biobanks, Biorepositories, Tissue Bank, Human Biospecimens, Human Samples, Patient Samples, Human Tissue Samples, Cancer Samples, FFPE Samples, Human Specimens, Human Serum Samples, Whole Blood Samples, Urine Samples, Peripheral Blood Mononuclear Cell, Plasma Samples.

FFPE Samples
Human Specimens
Human Serum Samples

Best office furniture in India by Lite Office said...

Good Blog, Such a Nice Content, Informative for Readers Keep Posting.

Visit us for
Best office furniture
furniture dealership
furniture franchise India

Packers and Movers In Nagpur said...

Thanks for sharing, information is really useful, keep posting.
Packers and movers in Bhopal
Movers and packers in Bhopal

Maruti movers and packers said...

Great Article.Thnks for sharing .I have been searching for such an informative post since many days and it seems my search jst ended here.Goodwork.Keep posting.


Packers and movers in Nagpur|Movers and packers in Nagpur
home shifting services in Nagpur

Anonymous said...

learn Quran online with highly qualified male and female Quran tutors.

Online Quran academy

VAANIL NILA said...

Need grab your top place in the ecommerce business field? Get in touch with the Ecommerce app development company, Ampro Techvalley. Ampro will fix your stable place in the ecommerce business field by giving its hopeful methods in your ecommerce apps. Ampro always do its best services by giving leading ideas to develop your online store. with this service, Ampro also gives custom software development service, android mobile app development service, and digital marketing service for creating your own image in the business field.

top social services said...

A social media panel, also known as an SMM pak panel or social media marketing panel, is an online platform that enables people or organisations to buy different social media marketing services to increase their online presence and engagement on well-known social media platforms. These platforms often consist of LinkedIn, Facebook, Instagram, Twitter, YouTube, TikTok, and others.



SMM Marketing said...

For your knowledge, the SMM (Social Media Marketing) panel is a website where you may buy various social media services, including likes, followers, comments, and views. Individuals, organizations, and digital marketers can boost their visibility on social media networks by using SMM panels.

insightfullnk said...

A crafted language, Evlwendz, emerges as a global tool, aiding people in writing and communication. Its growing impact as a writing assistant benefits users worldwide. Discover the capabilities of this language innovation, now available to enhance language proficiency.

John Fei said...

This article provides valuable insights into the critical importance of cybersecurity in the aviation industry, check https://attractgroup.com/blog/aviation-cybersecurity-strategies/ for full details. The detailed analysis of potential threats and mitigation strategies is both informative and eye-opening. It's clear that protecting sensitive data and ensuring the safety of passengers should be top priorities for all aviation companies. Great read!

DianApps said...

This is an insightful and informative article on mobile app development framework, this blog has given a detailed analysis of the framework necessary for developing a custom mobile app development.