MediaEval 2014: THU-HCSIL Approach to Emotion in Music Task using Multi-level Regression

Abstract

This working notes paper describes the system proposed by THU-HCSIL team for dynamic music emotion recognition. The procedure is divided into two module - feature extraction and regression. Both feature selection and feature combination are used to form the nal THU feature set. In regression module, a Booster-based Multi-level Regression method is presented, which outperforms the baseline significantly on test data in RMSE metric for dynamic task. This work was partially supported by the National Natural Science Foundation of China (No.61171116) and National Basic Research Program of China (973 Program, No.2012CB316401).

1. INTRODUCTION

The objective of Emotion in Music task in MediaEval 2014 Workshop is to predict 2-dimensional emotion of music for both the whole clip (static) and the sequenced 0.5-second segments (dynamic). For more details about the task and data set, see the task overview paper [ 1 ].

FEATURE SELECTION AND COMBINA

TION

This module is based on the 6552-dimensional EMO-LARGE features provided by organizers. These features are extracted by OpenSmile Toolbox [ 2 ] and are calculated by low-level features (LLDs) such as MFCC, Spectral, ZCR, loudness. Details of LLDs and statistic functions can be found in [ 2 ].

Since the redundancy and insu ciency of the base features, we use both feature selection and feature combination to improve our feature set.

Feature Selection

Firstly, two algorithms were used for feature importance ranking. The rst is Random Forest [ 3 ], which constructs a multitude of decision trees at training time and the importance of feature can be computed through permutation over all trees. The second is Extremely Random Trees (Extratrees) [ 4 ], another type of random trees with di erent nodesplitting strategy. Let FnRF denote the rst n important features in Random Forest and FnET in Extra-trees.

Then, we de ne the selected features as FnSLT = {FnRF ∩ FnET }. Using XGBoost lib [ 5 ], the optimized n is determined according to the RMSE performance of FnSLT . In our work, 661-dim FnSLT is obtained for arousal when n = 1280 and 522-dim for valence when n = 1280. 2.2

Feature Combination

Some music-related features, such as mode, timbre, are not included in the EMO-LARGE features, while they are proved to be important in music emotion recognition [ 6 ]. Thus, we use two toolboxes - MIRToolbox [ 7 ] and Bregman music toolbox [ 8 ] - with default settings to extract such additional features for our task.

For each 0.5 second segment, 217 features are extracted in total (see Table 1). Then we combine these supplement features with the selected features to form our nal THU feature set, 878 features (661+217) for arousal and 739 features (522+217) for valence. 3.

MULTI-LEVEL REGRESSION

With the assumption that the regression accuracy can be improved by reducing the scope of model, we propose a 2level regression method in this section. Figure 1 shows the framework of this method.

The rst-level regression is just a naive procedure commonly used in regression module. For x in test set X, we predict the test result r0 = f (x; RG) by model RG, a global model trained with entire training dataset XT r.

For each emotion dimension, we divide [−0:75; 0:75] (based on the ground truth distribution of XT r) into 6 bins with equal length (0.25). For x in X, its bin index i = Ind(x) = ⌈(f (x; RG)+0:75)=0:25⌉. The 2nd-level regression result of x Subset

Selection Test sample x

RG

Training R1L R2L 噯噯 R6L Testing is r1 = f (x; RiL), where RiL is the local model for the ith bin trained by the subset of training dataset XiT r, which is determined by XiT r = {x|x ∈ XT r and g(x) ∈ [ai; bi]}, where g(x) is the ground truth of x and the boundaries {ai; bi} are computed on dev set XD as follows:

We rst de ne Gi = {g(x)|Ind(x) = i; x ∈ XD}. After investigating the distribution of Gi, we then select ai = min{g|CDFi(g) ≥ 1:5%} and bi = max{g|CDFi(g) ≤ 98:5%}, where CDFi is the cumulative distribution function for Gi, to eliminate outliers and x the con dence level of dev set XD at 97%.

RESULTS AND CONCLUSIONS

Results in this section are all modeled on THU feature set and experiments are all for dynamic task. 4.1

Experiment Results

5-fold cross validation is used and is song independent (separated by song IDs randomly). 1/5 of training set is used as dev set (as mentioned in section 3).

Multiple regression algorithms were compared (Table 2). We choose tree booster in XGBoost for the following experiments, in which max tree-depth = 4, step size shrinkage = 0:1 and minimum loss reduction = 1:0.

Table 3 illustrates that the performance of multi-level regression system is improved slightly in both arousal and valence case. 4.2

Results on Required Test Data

Each clip from test data is split into 90 0.5-second segX Tr 1 X Tr 2 噯噯 X Tr 6

Booster Trainer ments and predicted in segment level. Table 4 shows the results of 5 runs submitted, run 1-4 are for dynamic task and run 5 is for static task. XGBoost lib is employed for regression in all runs.

• Dynamic task (subtask 2): Run 1. One-level regression.

Run 2. Two-level regression.

Run 3. Run 1 + smooth. For one clip, we replace segment result in Run 1 with the average of 4 adjacent segment results and itself as following:

rrun3(i) = mean(rirun21; :::; rir+u2n1); i = 3; 4; :::; 88. Run 4. Run 2 + smooth. • Static task (subtask 1):

Run 5. For each clip, average over 90 segment results from Run 2.

Result shows that our result outperforms baseline on RMSE metric. The lower APC results might have been as a result of not considering the correlation between adjacent segments. Performance of Run 3 and 4 on APC shows that the smooth module can increase the inter-clip correlation e ectively.

References

[1] A. Aljanaki , Y-H. Yang , M. Soleymani . Emotion in Music Task at MediaEval 2014 . In MediaEval 2014 Workshop, Barcelona, Spain, October 16 -17 2014 .

[2] Eyben , Florian, Martin Wollmer, and Bjorn Schuller. Opensmile: the munich versatile and fast open-source audio feature extractor . In Proceedings of the international conference on Multimedia. ACM , 2010 .

[3] Breiman , Leo. Random forests . In Machine learning 45.1 ( 2001 ): 5 - 32 .

[4] Geurts , Pierre, Damien Ernst , and Louis Wehenkel . Extremely randomized trees . In Machine learning 63.1 ( 2006 ): 3 - 42 .

[5] https://github.com/tqchen/xgboost/

[6] Yang , Yi-Hsuan , and Homer H. Chen . Machine recognition of music emotion: A review . In ACM Transactions on Intelligent Systems and Technology (TIST) 3 .3 ( 2012 ): 40 .

[7] Lartillot , Olivier, and Petri Toiviainen . A Matlab toolbox for musical feature extraction from audio . In International Conference on Digital Audio E ects . 2007 .

[8] http://bregman.dartmouth.edu/bregman/