The task of Violent Scenes Detection requires creating a system to detect segments which contain physical violence in both movies and videos found on the web, which is a very challenging task due to camera jitters in hand-shot videos and free shot boundary in movies and web videos. In this paper, we present a novel system by combining shot boundary detection, feature extraction in both audio and video domains, Bag-of-Words model and Support Vector Machine. The key part of system lies in trajectory based features that are calculated around robust optical ows. These ows are extracted by a novel salient keypoint trajectory algorithm. According to our results, a good detection performance can be achieved by using trajectory based features combined with dense SIFT and MFCC.
Violent Scenes Detection (VSD) is a challenging task which
requires teams to build a high performance system to
automatically detect video segments containing violence. VSD
2014 contains two di erent sub-tasks: main task and
generalization task. A brief introduction to the dataset for
training and testing as well as evaluation metrics of these two
sub-tasks is given in [
The architecture of the proposed system is shown in Fig. 1.
We adopt the Bag-of-Words (BoW) framework with
Gaussian Mixture Model (GMM), Fisher Vector (FV) and
Support Vector Machine (SVM). A threshold based video shot
boundary detector is rstly used to detect video shot
boundaries [
In VSD 2014, there are no video shot boundaries provided,
neither for movies nor web videos. This causes di culties
for feature extraction and encoding. In order to address this
issue, we employ the shot boundary detection method
presented in [
For feature extraction, two di erent kinds of video features are used including trajectory based features and one appearance feature. 2.2.1
Firstly, salient keypoint trajectories are generated to track 1 2 3 4 5
HOG,HOF,MBH
HOG,HOF,MBH
HOG,HOF,MBH
HOG,HOF,MBH
HOG,HOF,MBH
Dense SIFT
Dense SIFT
Dense SIFT
Dense SIFT
human actions at multiple spatial scales [
After the extraction of descriptors, these feature vectors are normalized with the signed square root, and then, PCA is individually applied to each of these three feature vectors (HOG, HOF and MBH) to reduce to half of the original dimension. Then, FVs are computed to construct a codebook for each descriptor. We compute one FV over the complete video, and apply signed square root normalization which is able to signi cantly improve the recognition performance in combination with linear SVM.
As far as classi cation is concerned, linear SVM is employed in this work and early fusion is performed to generate the nal feature vector by concatenating the aforementioned four feature vectors (HOG, HOF, MBH and dense SIFT) into a single one. In our implementation, the standard linear LIBSVM is used with the penalty parameter C equal to 100, which has shown to exhibit good performances. 2.2.2
Due to auditory clues in segments which contain violent
scenes, features of audio segments should be considered.
Therefore, we adopt the popular Mel-Frequency Cepstral
Coe cients (MFCC) algorithm [
The con guration of our submitted ve runs are summarized in Table 1. Regarding the late fusion, an arithmetic sum of scores outputted from SVM for video features (trajectory based features and appearance feature) and audio feature is calculated; for double fusion, rst we do early fusion of video features, and then late fusion of video and audio features. The weight setting segmented by colon in Table 1 stands for the weights applied to di erent kinds of features MFCC MFCC MFCC MFCC MFCC
Fusion Late Fusion Double Fusion Double Fusion Late Fusion Late Fusion
Weights during late fusion. 3.
We submit ve runs with the results given in Table 2 using the MAP2014 measure. The comparison of run1 and run4 show that the dense SIFT feature can help improve the recognition performance in the generalization task. However, there is a performance drop in the main task. The reason for this is that the late fusion strategy and weights assignment are sub-optimized for dense SIFT in the main task. By comparing run2 vs. run3 as well as run4 vs. run5, we conclude that di erent weights assignment will a ect the recognition performances, and the optimum weight setting di ers for di erent datasets. In general, we obtain better results in the generalization task than the main task. One reason for this is that the video shots in the generalization task do not change as frequent as that in the main task, which improves the performance of trajectory based features. It also indicates that the main task is more challenging than the generalization task.