Mood Player
- Details Hits: 15124
1. Introduction
Our project is an app that uses face detection and mood recognition to determine the user's mood and based on this, it gives a personalized playlist. The face detection algorithm is based on the OpenCV library, while the mood detection part will be based on pattern matching. Once we have the information needed, we'll use the last.fm database which links every song with tags that describe it.2. State of the Art
By doing research on Google Play, we didn't find an app which has the same functionality as what we want to do. Some of them will do only mood detection, others will create playlists based on a mood given by the user, but no application combines both of the functions.The majority of mood detection mobile apps use fingerprints, and very few use face detection, an example being Facescanner:
https://play.google.com/store/apps/details?id=com.wideapps.android.facemoodscanner&feature=search_result#?t=W251bGwsMSwxLDEsImNvbS53aWRlYXBwcy5hbmRyb2lkLmZhY2Vtb29kc2Nhbm5lciJd
Although we didn't expect perfect mood detection, Facescanner doesn't have a good functionality and it fails to recognize even obvious moods such as happy, sad, etc. Another problem with it is the big number of moods it tries to recognize, which will mislead the user.
For the playlist part, there are some good apps that give the expected results:
Stereomood
https://play.google.com/store/apps/details?id=com.mutado.android.stereomood&feature=search_result#?t=W251bGwsMSwxLDEsImNvbS5tdXRhZG8uYW5kcm9pZC5zdGVyZW9tb29kIl0.
Moodagent
https://play.google.com/store/apps/details?id=com.moodagent.android&feature=search_result#?t=W251bGwsMSwxLDEsImNvbS5tb29kYWdlbnQuYW5kcm9pZCJd
3. Software Requirements
This application is designed in order to generate a playlist according to the user`s mood and offers the following functionalities:-set your mood manually: happy/sad
-get your mood automatically by analyzing a periodical camera capture(frequency can be set from menu)
-set the music tempo, from calm to energetic
Having this data, the user emotion is given a specific tag and then using the last.fm database and the YouTube Api, a list of appropiate songs is loaded and played automatically. In this way, the user will have the song and also the YouTube video required.
Mood Player also offers the possibility to post the current playing song and a desired status on your own Facebook wall, only by logging in and clicking the share button.
4. System Design
"Mood Player" is an app which needs Internet Connection, as it loads and plays videos in real time.When first launching the application, the user has to take two photos of himself, a sad and a happy one, which will be later used as patterns for face detection.
"Mood Player" was designed as a single user app, being personalized with mood captures, but you can reset the initial photo set anytime.
The app uses the front camera to take pictures without the user noticing and creates a playlist based on the detected mood and the energy level which is set by the user. If the user doesn`t want to listen to a song according to his current detected mood, he can change it manually. This choice is useful if someone wants to change his negative emotional state by listening to some happy motivational songs.
5. Implementation
We have used the Android SDK from the Android Bundle to develop this project. Mood player is also based on three external sources: the openCV library, the facebook SDK and the youtube API for Android, so you will need to link these with the project if you want to build it by yourself.The source files are here: https://github.com/alinmindroc/facedetection
The project has four activities:
1. The face detection activity:
It's role is to take the pattern pictures for face detection.
It occurs only at the first use of the app, or when the user wants to reset the pattern pictures.
This is based mainly on a camera view that displays input from the front camera and detects the user's mouth. After taking the two pictures, we can go right to
the music player.
2. The music player activity:
You can see examples of the two modes of the player, the one on the left with face detection deactivated, and on the right, where face detect is checked, the app takes pictures of the user at a specified interval and it determines a mood for the user.
In the first mode, the user must select a mood and an energy level for the player to start and download a playlist.
The face detection works basically as follows:
- the app has two photos, with a sad (1) and a happy (2) expression of the current user. These are obtained in the first activity.
- at a certain moment, the app has a picture (3) of the user, from which it extracts the significant part. It could be the face, the eyes, or, as in our case, the mouth, as it can be used very easily to detect sad/happy moods.
- we get two floating point values by simply computing the norm of the pixel-by-pixel difference of the current mood(3) and each of the pattern images (1) and (2). This works
because the significant parts of the photos have small dimensions, so there can exist observable differences.
- we compare the norm of the differences, and the lower one sets the current mood(if the norm of the difference (1) - (3) is lower than the norm of (2) - (3), we have a sad mood).
For the playlist part, the workflow is:
- we have a mood returned by the face detection algorithm.
- we combine that with the energy level set by the user to create a tag and then an URL, for example http://www.last.fm/tag/happy%20hardcore/videos for the "happy hardcore" tag. Other tags are sad, happy, angry, calm, etc.
- we fetch the html source of the page, parse it and create a list of youtube IDs.
- we use the youtube api to generate a playlist based on these IDs.
The songs' titles are taken from the last.fm database, which associates a tag to every song, some tags being moods that we can use in the app, for example:
http://www.last.fm/tag/happy/tracks gives a full list of happy songs. Then, based on this list, we use the youtube api to create a music
playlist which you can browse in the embedded player. We have also included some songs on our own, to create some diversity.
3. The facebook activity:
This is opened if you want to share the song you are currently listening to on facebook. It is a simple login interface followed by a dialog where you can put some
extra text for the facebook post.
4. The hints/settings activity:
This is opened when an option is selected from the menu soft key. It allows the user to set the interval for face detection. It also gives some hints for the good use of the
application.