This paper presents the RECOD approaches used in the MediaEval 2014 Violent Scenes Detection task. Our system is based on the combination of visual, audio, and text features. We also evaluate the performance of a convolutional network as a feature extractor. We combined those features using a fusion scheme. We participated in the main and the generalization tasks.
The objective of the MediaEval 2014 Violent Scenes Detection task is to automatically detect violent scenes in movies and web videos. The targeted violent scenes are those \one would not let an 8 years old child see in a movie because they contain physical violence".
In this year, two di erent datasets were proposed: (i) a
set of 31 Hollywood movies, for the main task, and (ii) a
set of 86 short YouTube web videos, for the generalization
task. The training data is the same for both subtasks. A
detailed overview of the datasets and the subtasks can be
found in [
In the following, we brie y introduce our system and discuss our results1. 2.1
In low-level visual feature extraction, we extract SURF
descriptors [
Besides that, in order to incorporate temporal
information, we compute dense trajectories and motion boundary
descriptors, according to [
In mid-level feature extraction, for each descriptor type, we use a bag of visual words-based representation.
Furthermore, we use a visual feature extractor based on
Convolutional Networks, which were trained on the
ImageNet 2012 training set [
Using the OpenSmile library [
To represent the movie subtitles, we apply the bag of words approach: the most common, simple and successful document representation used so far. The bag of words vector is normalized using a term's document frequency.
Also, before creating the bag of words representation, we remove the stop words and we apply a stemming algorithm to reduce a word to its stem. 2.4
Classi cation is performed with Support Vector Machines
(SVM) classi ers, using the LIBSVM library [
In total, we generated 10 di erent runs: 5 runs for each subtask. For main task (m), we have: m1: 3 types of audio features + 3 types of visual features (including a visual feature extractor based on Convolutional Networks) + text features; m2: 1 type of audio features + 3 types of visual features (including a visual feature extractor based on
m3: 1 type of audio features + 3 types of visual features (including a visual feature extractor based on Convolutional Networks); m4: 1 type of audio features + 2 types of visual features + text features; m5: 1 type of audio features.
g1: 3 types of audio features + 3 types of visual features (including a visual feature extractor based on Convolutional Networks); g2: 1 type of audio features + 3 types of visual features (including a visual feature extractor based on Convolutional Networks); g3: 1 type of audio features + 2 types of visual features; g4: 1 type of audio features; g5: 1 type of visual features.
MAP 0.204 0.477 0.337 0.567 0.188 0.479 0.378 0.376 0.239 0.459 0.308 0.348 0.362 0.465 0.431 0.373 0.189 0.545 0.277 0.465 0.212 0.418 0.489 0.371 0.115 0.319 0.209 0.270 0.159 0.502 0.167 0.249 0.373 0.301 0.307 0.423 0.175 0.308 0.317 0.315
For the main task, our results are considerably below our expectations (based on our training results). By analyzing the results, we pointed out a crucial di erence between training and test videos. In the Violent Scenes Detection task, the participants are instructed in how to extract the DVD data and convert it to MPEG format. For the sake of saving disk space, we opted to convert the MPEG video les to MP4 or to M4V. However, that choice introduced a set of problems.
First, with respect to the training data, we converted the MPEG video les to MP4 or to M4V, depending on which video container we were able to successfully synchronize the extracted frames, regarding the numbers given by the 2Scenes are classi ed as violent or non-violent based on a certain threshold. groundtruth. Despite both containers store the video stream in H.264 format, we did not notice that the M4V conversion resulted in a di erent video aspect ratio (718 432 pixels). Similarly, the audio encoding was also divergent: MP3 audio for MP4, while AAC audio for M4V. Next, due to frame synchronization issue, we kept the test data in its original format (MPEG-2, 720 576 pixels, with AC3 audio). Therefore, we faced the problem of dealing with di erent aspect ratios in training and testing data, as well as distinct audio formats.
For the generalization task, the problem is alleviated because the test data is provided in MP4.
Tables 3 reports the uno cial (u) results for main task that we evaluated ourselves. Here, the results are obtained by using the data (training and test sets) in MPEG format. The rst column indicates which input features were used: u1 for 1 type of audio features and u2 for text features. Unfortunately, due to time constraints, we were not able to prepare more runs.
It should be mentioned rst that, the results for run u2, are independent of the video format, since we directly extracted the movie subtitles from DVD. For run u1, we can notice a considerable improvement of classi cation performance, from 0.315 (run m5) to 0.493 (run u1), con rming the negative impact of using distinct audio formats. We are currently investigating the impact on visual features. u1 u2 8mil 0.351 0.601 0.636 0.530 0.521 0.352 0.463 0.493 0.402 0.237 0.407 0.345 0.232 0.277 0.188 0.298 5.
This research was partially supported by FAPESP, CAPES, CNPq and Project \Capacitaca~o em Tecnologia de Informaca~o" nanced by Samsung Eletro^nica da Amazo^nia Ltda., using resources provided by the Informatics Law no. 8.248/91.