This paper presents the approach used by the RECOD team to address the challenges provided in the MediaEval 2015 Affective Impact of Movies Task. We designed various video classi ers, which relied on bags of visual features, and on bags of auditory features. We combined these classi ers using di erent approaches, ranging from majority voting to machine-learned techniques on the training dataset. We only participated in the Violence Detection subtask.
The MediaEval 2015 A ective Impact of Movies Task
challenged its participants to automatically classify video
content, regarding three high-level concepts: valence, arousal
and violence [
The activities of classifying video valence and of classifying
video arousal were grouped under the same subtask: the
Induced A ect Detection. The classi cation of violence, in
turn, was related to the Violence Detection subtask, where
participants were supposed to label a video as violent or not.
For both subtasks, the same video dataset was annotated
and provided. It consisted of short clips, extracted from
199 Creative Commons-licensed movies of various genres.
A detailed overview of the two subtasks, metrics, dataset
content, license, and annotation process can be found in [
In the following sections, we detail the classi ers we designed to solve the task. Thereafter, we explain the setup of the submitted runs, and report the results, with the proper discussion.
We designed video classi ers based on bags of visual features, and on bags of auditory features. Following the typical bags-of-features-based approach, these classi ers implement a pipeline that is composed by three stages: (i) lowlevel video/audio description, (ii) mid-level feature extraction, and (iii) supervised classi cation. These classi ers are then combined either in a majority-voting fashion, or in a machine-learned scheme.1 1As we are patenting the developed approach, a few technical aspects are not reported on this manuscript. 2.1
First of all, similarly to Akata et al. [
We developed two classi ers based on bags of visual
features. These classi ers di er from each other mainly with
respect to the employed low-level local video descriptors. We
have a solution based on a static frame descriptor
(SpeededUP Robust Features, SURF [
In the particular case of the SURF-based classi er, SURF descriptions are extracted on a dense spatial grid, at multiple scales. In the case of the space-temporal-based one, we apply a sparse description of the video space-time (i.e., we describe only the detected space-temporal interest points).
Prior to the mid-level feature extraction, for the sake of saving extraction time, we also reduce the dimensionality of the low-level descriptions.
In the mid-level feature extraction, for each descriptor
type, we use a bag-of-visual-words-based representation [
In the high-level video classi cation, we employ a linear
Support Vector Machine (SVM) to label the mid-level
features, as suggested in [
We developed three classi ers based on bags of auditory
features. Analogously to the visual ones, these classi ers
di er from each other with respect to the employed low-level
audio descriptors. We thus use the OpenSmile library [
Prior to the mid-level feature extraction, for the sake of saving extraction time, we also reduce the dimensionality of the low-level descriptions.
To deal with the semantic gap between the low-level
audio descriptions, and the high-level concept of violence, we
adapt a bag-of-features-based representation [
Finally, concerning the high-level video classi cation, we employ a linear SVM. 2.3
To combine various classi ers, we adopt two late fusion schemes.
In the rst one, we combine the scores returned by the various classi ers in a voting fashion. After counting the votes, we designate the video class as being equal to the most voted one. To attribute a nal score, we pick the score of the classi er that presents the strongest certainty regarding the video class.
In the second combination scheme, we concatenate the positive scores of the classi ers in a prede ned order, and feed them to an additional classi er. 2.4
External Data and Data Augmentation
In the dataset of this year, 6; 144 short video clips were
provided in the development (i.e., training) group [
Given that these web videos were, in average, longer than the videos of this year, we decided to segment the positive annotated chunks in parts of 10 12 seconds. That leaded to a total of 252 additional positive segments to augment our positive training dataset.
This year, participants were allowed to submit up to ve
runs for the violence detection subtask, with at least one
requiring the use of no external training data [
Table 1 summarizes the runs that were submitted this year to the competition. In total, we generated ve di erent runs. In two, we did not use external data, while on the remaining other three, we employed external data, as explained in Section 2.4.
MAP 1 2 3 4 5
No No
0.0690 Majority Voting Majority Voting Majority Voting Majority Voting
The best result (related to run 1 ) was achieved by the classi er that used a majority-voting late combination of visual and auditory features, trained with no external data (i.e., M AP = 0:1143). It performed better than the exact same solution (at run 3, with M AP = 0:1126), whose only di erence was the use of external data in the training phase (as explained in Section 2.4).
Therefore, we failed to augment the training data.
Reason for that may be related to the use of di erent types of
video sources, given that | in this year | Hollywood-like
movie segments were provided [
Notwithstanding, the majority-voting late combination of visual and auditory features indeed improved the classi cation performance. Although trained with the same videos (with external data), runs 4 (auditory only, with M AP = 0:0924) and 5 (visual only, with M AP = 0:096) achieved results that were below the combined solution (related to run 3, with M AP = 0:1126).
Regarding our results, in general terms, we did not have enough positive samples to learn a better classi er, a mandatory requirement of the machine learning techniques that we employed.
This paper presented the video classi ers used by the RECOD team to participate in the violence detection subtask of the MediaEval 2015 A ective Impact of Movies Task. The reported results show that a late combination of visualand auditory-feature-based classi ers lead to a better nal classi cation system, in the case of violence detection. Finally, given the machine leaning nature of our solutions, the challenging dataset of this year did not contain enough positive video samples to learn a better classi er, what strongly impacted on our results.
Part of the results presented in this paper were obtained through the project \Sensitive Media Analysis", sponsored by Samsung Eletro^nica da Amazo^nia Ltda., in the framework of law No. 8,248/91. We also thank the nancial support from CNPq, FAPESP and CAPES.