In this paper we describe the Imperial College London, Technische Universitat Munchen and University of Passau (ICL+TUM+PASSAU) team approach to the MediaEval's \A ective Impact of Movies" challenge, which consists in the automatic detection of a ective (arousal and valence) and violent content in movie excerpts. In addition to the baseline features, we computed spectral and energy related acoustic features, and the probability of various objects being present in the video. Random Forests, AdaBoost and Support Vector Machines were used as classi cation methods. Best results show that the dataset is highly challenging for both a ect and violence detection tasks, mainly because of issues in inter-rater agreement and data scarcity.
The MediaEval 2015 Challenge \A ective Impact of
Movies" comprises two subtasks using the LIRIS-ACCEDE
database [
In our work we have used both the baseline features
provided by the organisers [
The extended Geneva Minimalistic Acoustic Parameter
Set (eGeMAPS) was used to extract acoustic features with
the openSMILE toolkit [
For modelling the data we concentrated on two
out-of-thebox ensemble techniques: Random Forests and AdaBoost.
We used these two techniques as they are less susceptible
to the over tting problem than other learning algorithms
due to the combination of weak learners, they are trivial to
optimise as they have only one hyper-parameter, and they
usually provide close or on par results with the
state-ofthe-art for a multitude of tasks [
We submitted a total of ve runs. Run 1 consisted of predictions using the baseline features and the AdaBoost model. The predictions in runs 2 and 5 were obtained using the baseline plus our audio-visual feature sets and the Random Forest and AdaBoost classi ers, respectively. By looking at the distribution of labels in the development set, we observed that the most common combinations of labels are: 1) neutral valence (V n) and negative arousal (A ) (24%), and 2) positive valence (V +) and negative arousal (A ) (20%). Runs 3 and 4 are thus based on the hypothesis that the label distribution of the test set will be similarly unbalanced. In run 3 every clip was predicted to be V n, A+ and in Run 4 every one was V +, A . These submissions act as a sanity check of our own models, but also other competitors' submissions for this competition. 2.2
According to previous work [
Since the dataset is strongly imbalanced { only 272
excerpts out of 6,144 are labelled as violent { we up-sampled
the violent instances to achieve a balanced distribution. All
features were furthermore standardised with a z-score. As
classi er, we used the libsvm implementation of Support
Vector Machines (SVMs) [
We rst performed experiments with the full baseline feature set and found that the addition of the movie genre as feature improved the Mean Average Precision (MAP) from 19.5 to 20.3, despite degrading the UAR from 72.3 to 72.0. Adding our own audio-visual features provided a jump in the performance with the MAP reaching 33.6 and UAR 77.6. Because some movie excerpts contain partly relevant acoustic information, we empirically de ned a threshold on loudness based on the histogram, to exclude frames before computing the functionals. This procedure has improved the MAP to 35.9 but downgraded the UAR to 76.9. A ne tuning of the complexity parameter and coe cient yielded the best performance in terms of UAR with a value of 78.0, but slightly deteriorated the MAP to 35.7.
We submitted a total of ve runs. Run 1 { baseline features; Run 2 { all features mentioned above (except movie genre) with loudness threshold (0.038); Run 3 { same as Run 2 plus the inclusion of movie genre; Run 4 { as Run 3 but with a ne tuning of the hyper-parameters; Run 5 { similar to Run 3 but with a higher threshold for loudness (0.078).
Our o cial results on the test set for both subtasks are shown in Table 1.
Subtask 1. Our results for the a ective task indicate that
we did not do much better than was expected by chance for
arousal classi cation, and did slightly better than chance
for valence in run 5; we thus refrain from further
interpretation of results. This can be explained by the low quality of
the provided annotations for the dataset. The initial
annotations had a low inter-rater agreement [
Subtask 2. Results show that there is an important overtting in our models as the performance is divided by a factor of 2 between development and test partitions. This is, however, not really surprising since only 272 instances labelled as violent were available as training data. Moreover, the labelling task being performed not at the frame level but rather at the excerpt level does not allow to model precisely the information that is judged as violent, making the task highly challenging. We can nevertheless observe that the proposed audio-visual feature set brings a large improvement over the baseline feature set { the MAP is improved by a factor superior to 2, and that the inclusion of the movie genre as additional feature also allows a small improvement in the performance.
We have presented our approach to the MediaEval's
\Affective Impact of Movies" challenge, which consists in the
automatic detection of a ective and violent content in movie
excerpts. Our results for the a ective task have shown that
we did not do much better than a classi er that is based
on chance, although we use features and classi ers that are
known to work well in the literature for arousal and valence
prediction [
The research leading to these results has received funding from the EC's Seventh Framework Programme through the ERC Starting Grant No. 338164 (iHEARu), and the EU's Horizon 2020 Programme through the Innovative Action No. 644632 (MixedEmotions), No. 645094 (SEWA) and the Research Innovative Action No. 645378 (ARIA-VALUSPA).