This paper describes the music emotion recognition system developed at the University of Aizu for the Emotion in Music task of the MediaEval'2013 benchmark evaluation campaign. A set of standard feature types provided by the Marsyas toolkit was used to parametrize each music clip. Arousal and valence are modeled separately using Gaussian Process regression (GPR). We compared performances of the GPR and Support Vector regression (SVR) and found out that GPR gives better results than SVR for the static per song emotion estimation task. For the dynamic emotion estimation task GPR had some scalability problems and fair comparison was not possible.
Gaussian Processes (GPs) [
Database used in this evaluation is described in detail in
the Emotion in Music task overview paper [
3.
Dimensional music emotion recognition can be easily decomposed into two independent classical regression problems: one for the valence, and another for the arousal. Thus, our system consists of two regression modules and a common feature extraction module. 3.1
Features are extracted only from the audio signal which is rst downsampled to 22050 kHz. We tried various standard features tailored for music processing such as MFCC, Statistical Spectrum Descriptors (SSD), Chroma, Spectral Crest Factor (SCF), and Spectral Flatness Measure (SFM) separately as well as combinations of several of them. All feature vectors were calculated using the Marsyas toolkit with 512 samples frames with no overlap. For the dynamic emotion estimation task, rst order statistics (mean and std) of the feature vectors are calculated for a window of about 1 sec. giving 45 vectors per musical clip. For the static emotion estimation, same statistics for these 45 vectors are calculated resulting in a single high dimensional feature vector per song. After extensive preliminary experimentation we found that the best performing combination of features for the per song emotion estimation is MFCC, SCF, and SFM. Adding SSD features did not have any noticeable e ect, and Chroma features actually hurt the performance. We refer to this combination of features as UoA features. We have also experimented with features released by the benchmark organisers which we call MediaEval features. 3.2
Valence and arousal are modeled by separate GPR. We used standard Gaussian likelihood function which allows exact inference to be performed. The GP mean was set to zero and only the type of covariance kernel was varied. We experimented with the following kernels:
Linear (LIN): k(x; x0) = (xT x0 + 1)=l2 Squared Exponential (SE): k(x; x0) = x0)T (x x0)=2l2) Rational Quadratic (RQ): k(x; x0) = x0)=2 l2) 2(1+(x x0)T (x Matern 3 (MAT3): k(x; x0) = p3(x x0)T (x x0)=l2) 2(1 + r) exp( r); r = where and l are parameters learned from training data. Sums or products of several kernels are also valid covariance functions and often give better performance than single kernels.
First, we present our results on the development data obtained after 7-fold cross validation. In addition to GPR, results from SVR using the same conditions are given in the following tables. In the SVR case, the parameter C was manually optimized using a grid search in the range [0.01,100], and kernel parameters are set to their default values (using LIBSVM package) since they cannot be learned.
Table 1 shows the result for the static emotion estimation in terms of R2 metrics for both MediaEval and UoA feature sets. The last row of each feature set type shows the best performing combination of GPR covariance kernels.
Kernel Valence MediaEval features
Linear 0.112 RBF 0.017 LIN 0.132 SE 0.142
RQ 0.150
MAT3 0.143 LIN+RQ 0.170
UoA features Linear 0.314 RBF 0.367 LIN 0.322 SE 0.375
RQ 0.430 MAT3 0.395 SExRQ 0.437
Arousal 0.300 0.028 0.565 0.590 0.562 0.590 0.581 0.604 0.653 0.603 0.656 0.662 0.668 0.671 Algorithm
SVR GPR SVR
GPR
In Table 2, we summarize results of the dynamic emotion estimation task where Kendal measure is calculated after pooling all arousal or valence estimates from all songs together. We have to mention that since in this task the amount of data was 40 times bigger, we ran into some scalability problems with the GPR implementation and had to resort to approximations of the kernel matrix using much less data which, of course, decreased the performance noticeably.
Table 3 presents the results of the UoA submission runs: two for the static and one for the dynamic emotion estimation tasks. They are obtained using GPR with corresponding best performing kernels. Direct comparison with Tables 1 and 2 is possible only for RSQ lines and it can be seen that in contrast to MediaEval, UoA features give similar results. cial results on the test data obtained 5.
We described the UoA emotion recognition system for the "Emotion in Music" task of the MediaEval'2013 benchmark evaluation which is based on the Gaussian Process regression algorithm. Compared to the Support Vector regression, GPR has several advantages, such as truly probabilistic prediction, and ability to learn hyperparameters from data. Performance wise, the GPR achieved better results for the static per song emotion estimation, but failed for the dynamic emotion estimation due to some scalability problems.