In this working note, we propose a set of features and a classi cation scheme for detecting automatically violent scenes in movies. The features are extracted from audio, video, and subtitles modalities of the movies. In violent scenes classication, we found the following features relevant: the short time audio energy, motion component, and shot words rate. We classi ed the shots into violent and non-violent using nave Bayesian, Linear Discriminant Analysis (LDA), and Quadratic Discriminant Analysis (QDA) targeting to maximize the precision of the detection in the rst two minutes of retrieved content.
Visual media is nowadays full of violent scenes. Therefore, multimedia content is rated to protect minors or warn the viewers for graphic or inappropriate images.
Manual rating of all existing content is not feasible for the
fast growing digital media. An automatic method that can
detect violence in movies, including verbal, semantic and
visual violence can help video on demand services as well as
online multimedia repositories to rate their content.
In this task, we have used audio, visual and text
modalities to detect violent scenes in movies at shot level. Despite
its importance, this problem has not been extensively
addressed in literature. Giannakopoulos et al. combined
visual and audio features to design a multi-modal fusion
process [
2.1.1
The extracted audio features are: energy entropy, signal
amplitude, short time energy, zero crossing rate, spectral ux,
spectral rollo . A more detailed description can be found in
[
The subtitles available for all DVDs carry a semantic
information of the movie content. We have parsed the le
content in a set of CaptionsElement where CaptionsElement is
an object having the four attributes (num, startTime,
stopTime, stemmedWords). The attribute num corresponds to
the dialogue position in the subtitles le content.
Each dialogue text is rst tokenized. The English stops
words were rst removed. Then, we used WordNet [
All the proposed content-based features have been extracted
using shot boundaries provided by MediaEval [
Three di erent classi ers were applied to detect violent shots, namely, QDA, LDA and nave Bayesian classi er. A postprocessing on the results of QDA was also done to consider the temporal correlation between consecutive shots. The post-processing consists on smoothing the con dence scores for the violent class from QDA using weights found from the transition probabilities on the training set.
According to the requirements, we have generated ve di erent runs trying di erent classi ers with prior probabilities. These characteristics are listed in Table 1. 3.1
The goal of the violence detection in the proposed use case scenario is to provide the user with the most violent shots in the movie. We de ned an evaluation criteria based on this use case scenario as follows. The detected violent shots were rst ranked based on their con dence scores, the rst 2 minutes on the top of the list were set aside as the retrieved content. The precision, recall and F1 score were then computed for the top ranked two minutes shots. We used a K-fold cross validation with K = 11 and di erent prior probabilities for each class. The best performance was achieved using LDA and QDA methods with equal prior probabilities for both classes. 3.2
The results of the last run correspond to the post-processing of the fourth run. A weighted average of the con dence scores was used to smooth the violent shots' decisions. The weights are given in Table 2 where the rst row represents the probabilities of transition to a violent shot while the four neighbouring shots are non-violent. The second row of the table represents the transition probabilities for transition to a violent shot while the neighbouring shots are violent. These values were obtained from the training set. The postprocessing reduced the false positives signi cantly. We ultimately obtained the best result with the with minimum MediaEval cost C 2:02 and recall r = 0:87 (Table 3) using LDA with prior probabilities 0.3 and 0.7 respectively for non-violent class and violent. However, if we look at both F1 score and MediaEval cost the fourth run which was with QDA and equal prior probabilities performed better. These results matched our expectations from the cross validation results on the training set.
We have proposed a set of features to automatically detect violent material at shot level for commercial movies. The performance of the proposed system have been evaluated based on a detection cost function weighting by false alarms and missed detections rate. The short time energy, the motion component and the shot words rate are proposed and used as relevant features to classify a movie's shots as violent or non-violent. The proposed methods were unable to detect all the violent scenes without sacri cing the false positive rate. This is due to the fact that the proposed features are not enough to capture all violent actions or events. Automatic detection of more high level concepts such as scream, explosion, or blood are needed to improve the detections. 5.
This work is supported by the European Community's Seventh Framework Programme [FP7/2007-2011] under grant agreement Petamedia No. 216444.