This paper describes the team MTM participation in Violent Scenes Detection (VSD) task of the MediaEval 2014 campaign. We propose an approach to the problem of detecting violence, which is based on probabilistic graphical models using Mel-frequency cepstral coefficients (MFCCs) as audio feature. In our approach, we employ Dynamic Bayesian Networks (DBNs) to represent a violent scene as an dynamic system.
The goal of the Violent Scenes Detection (VSD) task of the Me-diaEval 2014 benchmarking campaign is to detect violence in movies [5]. This year the organizers of the VSD task released two datasets: (i) a set of 31 Hollywood movies, where 24 are used for training and 7 for the testing (our focus); (ii) Youtube set, composed of 86 violent and non-violent videos. Violence is defined as "one would not let an 8 years old child see in a movie because it contains physical violence". A model based on the variable-duration hidden Markov model is proposed to detect complex events using latent variables in Internet videos [6]. The authors of [1] propose an audio-visual approach to video genre classification using content descriptors that exploit audio, color, temporal, and contour information and demonstrated good results over other existing approaches by using a combination of these descriptors in genre classification. In [2], temporal structure of broadcast tennis video is recovered from HMMs. This trained HMM is used to analyze the temporal interleaving shots.
We propose to model video based on temporal structure and principle of causality using Dynamic Bayesian Networks (DBN).
For this year's benchmark, we have developed an acoustic system based on temporal data (MFCC vector). The main idea behind this approach is to represent a violent scene as a dynamic system.
A DBN (see Figure 1) is a state-space model of random variable Vt [3]:
where Ut represents the hidden, Xt the input and Yt the output variable. A pair (B1, B2) defines a DBN, where B1 and B2 are
Copyright is held by the author/owner(s).
MediaEval 2014 Workshop, October 16-17, 2014, Barcelona, Spain
Figure 1: A graphical-model view of an DBN unrolled for 4 slices with hidden state sequence U and a observed node X.
BNs. The two-slice temporal Bayes net B2 (DBN unrolled for 2 slices), defines P (Vt|Vt−1):
where P a(V t i ) are the parents in the net. Next, our acoustic feature detector is described.
Our audio concept detector is based on MFCCs. The audio signal is segmented into acoustic frames with overlapping. Acoustic frames are used to group samples using a window with fixed length. We split the audio signal into frames of 40ms length, with 20ms overlap, and apply a Hamming window to each frame. The Hamming function is given by:
For each audio frame, 12 MFCCs (range 133Hz-6855Hz) and their first and second derivates are computed to build an acoustic vector y j :
After the feature extraction, a way of representing audio is through a feature vector model using Bag of Audio Words (BoAW). In this representation, each vector has the size of the vocabulary, where each vocabulary word represents a position vector. The i th vector value for a n audio segment equals the number of occurrences of that word i in the audio segment.
For each run, a naive DBN is trained using two different observed vectors Yt: (i) acoustic vector y j , and (i) BoAW by j with 128 audio words (see Figure 2). The likelihood of a model M , P (y1:T |M ), is used to assign a sequence y1:T to non-violent or violent label as follows:
M * (y1:T ) = arg max M P (y1:T |M )P (M ).
(
The Bayes Net Toolbox for Matlab (BNT) [4] is used to train the dynamic networks.
Table 1 shows the Mean Average Precision (MAP): MAP2014 and MAP@100 for the test movies. DBN with BoAW and DBN without have similar performances. Both approaches (run #1 and run #2) fail at detecting of violent scenes in the movie "8 Mile". The run #2 results are higher in the movies "BRAVEHEART", "DES-PERADO", "GHOST IN THE SHELL" and "V FOR VENDETTA", but lower for the movies "TERMINATOR 2" and "JUMANJI" in comparisom with run #1 (using MAP@100 and MAP2014 metrics). Run #2 uses BoAW representation, that has less observations (temporal segments) than run #1 approach, which uses directly the acoustic feature vector built from MFCCs. Our best result is 16.51% (MAP@100) or 2.64 % (MAP2014 ) for run #2 (see Table 2). We investigated the results and came to the presumption that BoAW removes noisy observations,while reducing the number of observations per segment. It might be related with the observation "grouping" when the BoAW is computed for the temporal segment (see Figure 2). Thus, BoAW removes data noise and builds a better representation for a scene (model observation). However, the results are still very poor. We suppose it could be due to features, only MFCCs seems not capable of distinguishing all violence and non-violence segments and generalize the violence concept. Further work directions relies in capture the causality in violence segments using different structures and other feature modalities (feature selection).
This work was supported in part by two grants from CAPES and CNPq.