This working note reports approaches of team KART to MediaEval2017 AcousticBrainz Genre Task and their results. To solve the problem, we mainly considered the sparsity and noise of data, network design for the multi-label classification, and implementation of successful Deep Neural Network (DNN) models. We propose three steps of preprocessing and depict two diferent approaches: a single-column model and a multi-column model.
A music genre is a class, type or category that defined by
convention [
We approached the problem based on careful consideration of following: • How to handle the noisy and sparse data? • How to solve the multi-label classification task? • How to apply a variety of successful deep neural network models to our task?
Before starting the model training, we conducted three steps of feature preprocessing: (i) feature vectorization, (ii) fitting outlier feature values to the outlier boundaries, and (iii) selecting features by feature value distribution analysis.
Essentially, we tried to use features as raw as possible. Its underlying assumption is that the deep neural network model can learn useful representations of the raw data if there are suficient amount of samples. We omitted all the information under ’meta data’ keys. Further, we applied PCA to the covariance matrices of filter banks. For the computation convenience, we only choose the eigenvector whose corresponding eigenvalue is most large. In addition, we encoded categorical features into binary vectors in one-hot manner.
For some features, there are outliers with extremely high or low values in comparison with their medians (Figure 1 left). We ∗Author’s names are listed alphabetically; authors contributed equally to this work. suppressed or boosted these outlier values to the outlier boundaries. A lower limit and an upper limit boundaries for judging outliers are defined as :
Lower Limit = FirstQuartile − 1.5 ∗ IQR U pper Limit = T hirdQuartile + 1.5 ∗ IQR (1) where Inter Quartile Range (IQR) is a diference between the first quartile and the third quartile. Values bigger than the upper limit were clipped at the upper limit, and values smaller than the lower limit were boosted to the lower limit. Figure 1 (right) shows the distribution after the fitting of the Figure 1 (left).
Upper limits and lower limits of the training set were derived from feature value distributions of each feature and each genre. However, since the genre labels of the test set should not be informed, we thresholded the test set at the maximum upper limit and the minimum lower limit of the training set.
After the outlier fitting, we defined features that concentrate around same values for every genre as useless features and removed 133 features from the training and test sets. 3
We implemented two Feed-forward Neural Networks (FNNs). The main diference in architectures is whether the label hierarchy between the genre and the sub-genre is considered explicitly. Since the provided input features are already processed, the model is designed for encoding interdependency among labels. 3.1
As a baseline, we implemented an Single Column FNN (SCNN) whose output dimensions correspond to the entire labels. The label hierarchy between genre and sub-genre was not considered in this model. The genre and sub-genre are equally treated as independent labels. We applied the weight vector w with the loss function, to give less penalty for more frequent labels as following: wi = 1 +
1 log (1 + fi ) where fi denote raw count of label i in the given training dataset. In this way, the error from less frequent labels can be counted relatively larger than more frequent labels. It leads to learning less frequent labels more sensitively. The loss function of this model is: 1 Õ LSC N N = M m,i wi H ( yˆi(m), yi(m)) where H denotes the binary cross-entropy of the true label and the prediction, yi(m) is a binary vector for label i of the observation m, yˆ (m) is a prediction for label i of the observation m, and M denotes i the size of the mini-batch. The inference of yˆi(m) is:
yˆ (m) = f (x(m); θ )
where f (x(m); θ ) denotes an FNN that has a set of model
parameters denoted as θ , and x(m) is a feature vector of the observation
m. We applied ReLU[
During the inference, we only predict labels whose probability exceed the threshold α . We set α as 0.2 and 0.3 for SCNN, which are found as optimal values by the cross-validation. 3.2
To explicitly reflect label dependency between sub-genre and genre, we implemented a Multi Column FNN model (MCNN). It has a parallel structure for each set of sub-genre and genre, and merged by the Bayes rule on the top layer, as following:
yˆ д(m) = f (x(m); θд ) yˆ s(mд) = yˆ д(m∗ ) f (x(m); θsд ) where yˆ д(m) and yˆ s(mд) denote the estimated probabilities of genre and sub-genre from each model. The posterior probability of subgenre yˆ s(mд) is conditioned by yˆ д(m∗ ). Here д∗ denotes the genre where the sub-genre sд belongs. The loss function of this model is: 1 Õ LMC N N = M m
[wдH (yˆ д(m), yд(m)) + wsдH (yˆ s(mд), ys(mд))] where wд and wsд are scalar weights to balance learning rate of the genre column and the sub-genre column. We used the ratio of 1:9 between wд and wsд , considering sub-genre labels are more sparse than genre labels. We used the batch normalization only after the input layer. The dropout was not applied. We used threshold αsд = 0.25 for sub-genre and threshold αд = 0.4 for genre, respectively.
Assuming given feature set is already suficiently processed, we
also applied a highway network[
When comparing the scores, we noticed that SCNN is overall better than Baselines, but the MCNN is better than Baselines only in per label scores. However, considering the recall scores, which are not presented in the note due to space, it shows that both suggested models score better recall than the Baseline 2. This shows both models are working better for predicting sparse sub-genres than baselines and suggesting our weighted losses work as intended.
Compared to the validation accuracy, the test accuracy got worse. Since the training set and the validation set are skewed and sparse data, our models failed to learn generalized parameters. Experiments with the data augmentation have to be explored to overcome the drawback.
Also, a large model size of MCNN can be another reason of its worse test accuracy. The highway networks have 2 times larger than the standard fully-connected layers and MCNN has two columns of the network to model genre and sub-genre predictor separately. This structure makes the model 4 times bigger than SCNN model. A Multi-Column architecture with small units and standard fullyconnected layers will be useful.
Content-based Music Genre Recognition from Multiple Sources