In this paper, we describe the music emotion recognition system named as JU_NLP to find the dynamic valence and arousal values of a song continuously considered from 15 second to its end in an interval of 0.5 seconds. We adopted the feed-forward networks with 10 hidden layers to build the regression model. We used the correlation-based method to find out suitable features among all the features provided by the organizer. Then we applied the feedforward neural networks on the above features to find out the dynamic arousal and valence values.
Rapid growth of internet has been experienced all over the world
since past ten years. It also expedites the process of purchasing
and sharing digital music in the Web. Thus, such a large
collection of digital music needs an automated process for their
organization, management, search, playlists generation etc. People
are more interested in creating music library that allows them to
access songs in accordance with their moods compared to the title,
artists and/or genre [
Emotion in Music Task at MediaEval addresses the problem
of automatic emotion prediction of music in a time frame of 0.5
second as we can observe the significant emotional changes
during the discourse of a full length song. The organizers
provided the annotated music clips for the Music Emotion
Retrieval (MER) task. The music clips were annotated via
crowdsourcing using Amazon’s Mechanical Turk1 (MTurk) [
On the other hand, correlation is used to reduce the feature dimension. If we treat all the features and the class in a uniform manner, the feature-class correlation and feature-feature intercorrelations may be calculated as follows.
! =
1
!"
+ ( − 1)!!
where ! is heuristic metric of a feature subset s
containing k features, k is the number of components, !" is the
average feature–class correlations, !! is the average
featurefeature inter-correlation [
From the above equation, we can calculate how predictive
one attribute is with respect to another. A collection of instances is
considered pure if each instance is the same in contrast to the
value of a second attribute; the collection of instances is impure
(to some degree) if instances differ with respect to the value of the
second attribute. To calculate the merit of a feature subset using
above equation, feature-feature inter-correlations (the ability of
one feature to predict another and vice versa) must be measured as
well [
Subtask 1: In this subtask, a fixed feature set has been provided by the organizers and we have to implement the models of our choice to identify the valence and arousal for the clips captured in 0.5 second time interval. In this work, we employed two feed-forward neural networks based regression model, in order to map feature values to the arousal and valance scores. Both of the feed-forward neural networks use the same set of feature values but are respectively trained on the arousal or valence score. Each of the feed-forward neural networks is employed with 10 neurons in the hidden layers. We divided the whole training set into 5 parts to reduce the computation time, i.e. we trained our system using around 5000 instances at a time. Then, we tuned our system using a single portion as training and another portions for testing. We calculated the Mean Square Error (MSE) for each of the training sets. Finally, we tested the whole test dataset using five trained modules and got five sets of results. These five sets of results are combined using average and inverse weighted average technique. In the average technique, we simply took the average of all the five results. Whereas, the inverse weighted average is calculated as the equation below, ! Output!"#$!!"# = ! 1! (2)
! !!
Where ! is the ith output and ! is the MSE of the ith module. From the equation, we can see that we gave less priority to the result, which has derived from the module having the maximum MSE.
Finally, the Root-Mean-Square Error (RMSE) is used to evaluate the MER systems. We also reported the Pearson’s correlation (r) of the prediction and the ground truth. The final RMSE and ‘r’ for the above two systems (named as baseline feature system with our model average and weighted average) were given in the Table 1.
Subtask 2: In this subtask, a fixed regression model has been provided by the organizers to develop the MER systems based on the different features of our choice. According to the literature, we found that most of the important features are provided by the organizer as the baseline features. So, we focused to find the important feature rather than finding any extra feature. Thus, we used the correlation based feature reduction technique to reduce the baseline feature set given by the organizers as per the formula of correlation in equation 1.
We found 70 and 114 numbers of important features using this correlation formula for the arousal and valence, respectively. Later on, we used the restricted correlation and found 24 and 70 numbers of important features for the arousal and valence, respectively. We also selected 28 important features for valence.
Subtask 3: In this subtask, we implemented the feed-forward neural network on the derived features using correlation in order to build the MER system. We built five systems for difference sets of arousal and valence features described in subtask 2. The RMSE and ‘r’ values for the above five models were shown in Table 1.
We used feed-forward neural network to develop a regression based system to find the dynamic arousal and valence for analyzing the emotion in music. The correlation method is used to reduce the feature dimension in order to find suitable features for both arousal and valence. The best model yields minimum RSME of 0.2622 and 0.2913 for arousal and valence respectively using 70 best features, but the ‘r’ value for the arousal was high as compared to other system for arousal. In future, we want to explore deep learning neural networks for music emotion recognition.
The first author is supported by Visvesvaraya Ph.D. Fellowship funded by Department of Electronics and Information Technology (DeitY), Government of India. The authors are also thankful to the organizers A. Aljanaki, Y. Yang and M. Soleymani for their support and help. BaF+ OM (Weighted Average)
OF (24) + OM OF (70) + OM OF (114) + OM OF(28) + OM
0.2689 ±0.1073 ±0.1062 ±0.1011 ±0.0899 X X 0.4678 0.4671 ±0.2307 ±0.2282 ±0.2531 ±0.2489 X X
±0.1612 ±0.1627
X ±0.1452 ±0.1666
X ±0.0281 ±0.3312