=Paper=
{{Paper
|id=None
|storemode=property
|title=NII, Japan at MediaEval 2011 Violent Scenes Detection Task
|pdfUrl=https://ceur-ws.org/Vol-807/lam_NII_Violence_me11wn.pdf
|volume=Vol-807
|dblpUrl=https://dblp.org/rec/conf/mediaeval/LamLSD11
}}
==NII, Japan at MediaEval 2011 Violent Scenes Detection Task==
https://ceur-ws.org/Vol-807/lam_NII_Violence_me11wn.pdf
NII, Japan at MediaEval 2011
Violent Scenes Detection Task
Vu Lam Duy-Dinh Le
University of Science National Institute of
227 Nguyen Van Cu, Dist.5 Informatics
Ho Chi Minh, Vietnam 2-1-2 Hitotsubashi,
lqvu@fit.hcmus.edu.vn Chiyoda-ku
Tokyo, Japan 101-8430
ledduy@nii.ac.jp
Shin’ichi Satoh Duc Anh Duong
National Institute of University of Science
Informatics 227 Nguyen Van Cu, Dist.5
2-1-2 Hitotsubashi, Ho Chi Minh, Vietnam
Chiyoda-ku daduc@hcmus.edu.vn
Tokyo, Japan 101-8430
satoh@nii.ac.jp
ABSTRACT effective than using local features and combination of global
We present a comprehensive evaluation of performance of features and local features can help to improve the overall
visual feature representations for MediaEval 2011 - Violent performance.
Scenes Detection Task. As for global features, color mo-
ments, color histogram, edge orientation histogram, and lo- 2. FEATURE EXTRACTION
cal binary patterns are used. As for local features, keypoint We evaluate both global features and local features. The
detectors such as Harris Laplace, Hessian Laplace, Harris global features include color moments, color histogram, edge
Affine, Dense Sampling are used to extract keypoints and orientation histogram, and local binary patterns. The local
SIFT and COLOR SIFT are used as descriptor of the re- feature is based on the BOW model in which the SIFT de-
gion around these keypoints. The results obtained by our scriptor is extracted at interest points detected by Harris
runs are presented. The demo is available at: Hessian Laplace and multi-scale dense sampling detector.
http://satoh-lab.ex.nii.ac.jp/users/ledduy/Demo-MediaEval
3. FEATURE CONFIGURATION
Keywords
3.1 Granularity
semantic concept detection, local features, global features,
Since global features do not capture spatial information,
sift, color sift
to overcome this problem, a grid n × m is usually used to di-
vide the input image into non overlapping sub-regions. The
1. INTRODUCTION features extracted from these regions are concatenated to
We have developed NII-KAORI-SECODE, a general frame- form the feature vector for the image.
work for semantic concept detection, and used it to partic-
ipate several benchmarks such as IMAGECLEF, MEDIAE- 3.2 Color space
VAL, PASCAL-VOC, IMAGE-NET and TRECVID. The Local binary patterns and edge orientation histogram are
purpose is to evaluate performance of various visual fea- extracted from gray scale image. For color moments and
ture representations for concept detection-like task. In this color histogram, color spaces including HSV, RGB, Luv, and
framework, first features are extracted from keyframes, then YCrCb are used.
concept detectors using these features are learned by using
SVM with χ2 RBF kernel. The probability output scores of 3.3 Quantization
the learned concept detectors are used for ranking. We con- For color histogram, we only use 8-bin histogram for each
sider the Violent Scenes Detection Task [1] as a kind of con- channel. For edge orientation histogram, we quantize ori-
cept detection task and use NII-KAORI-SECODE frame- entations into histograms of 12+1 bins, 18+1 bins, 36+1
work for evaluation of performance of global and local fea- bins, and 72+1 bins. For local binary patterns, we quantize
tures. The results show that using global features are more binary patterns into histograms of 10, 30, and 59 bins.
Each combination of feature type, granularity, quantiza-
tion, and color space forms one feature configuration. The
Copyright is held by the author/owner(s). feature configurations evaluated in this study are described
MediaEval 2011 Workshop, September 1-2, 2011, Pisa, Italy in Table 1.
Table 1: Feature configurations.
Quantization
Feature Type Granularity Color Space (#Bins) Total Configs
2x2, 3x3, 4x4, HSV, Luv,
Color moments (CM) 5x5, 6x6 RGB, YCrCb 3x3 20
2x2, 3x3, 4x4, HSV, Luv,
Color histogram (CH) 5x5, 6x6 RGB, YCrCb 8x3 20
Local binary 2x2, 3x3, 4x4,
patterns (LBP) 5x5, 6x6 GRAY 10, 30, 59 15
Edge orientation 2x2, 3x3, 4x4,
histogram (EOH) 5x5, 6x6 GRAY 12, 18, 36, 72 20
Local features
(harhes, harlap, heslap,
haraff, hesaff,
dense, phow
csift, sift, oppsift,
rgbsift, rgsift) 1x1, 2x2, 1x3, 3x1 GRAY 500 visual words 44
4. CLASSIFIER LEARNING
LibSVM is used to train SVM classifiers. The extracted
features are scaled to [0, 1] using the svm-scale tool of Lib-
SVM. The χ2 RBF kernel is used as similarity measure. The
optimal (C, g) parameters for learning SVM classifiers are
found by conducting a grid search with 5-fold cross valida-
tion on a subset of 3,000 samples stratified selected from the
original dataset.
5. EXPERIMENT
For each shot, one keyframe is used for training and test-
ing. The set of keyframes are provided by the organizer. To
generate training data, shots falling into positive segments
are considered as positive shots. The other shots are consid- Figure 1: ROCCurve of the submitted runs.
ered as negative shots. We apply the trained classifier to the
keyframes of the test set. The output scores of keyframes
are considered as scores of shots and used for ranking. We 6. DISCUSSION
use the threshold θ = 0.02 for binary decision.
We submitted 6 runs and the details of performances are • Using one keyframe per shot for training and testing
shown in Table 2. AED cost is the cost defined by the task’s needs to be reconsidered since in many cases, keyframes
organizer and MAP is mean average precision. The results are not related to violent scenes. We tried using 10
show that using global features are more effective than using keyframes per shot for training and testing. However,
local features and combination of global features and local the performance is even worse than that of using only
features can help to improve the overall performance. one keyframe. The reason could be multiple keyframes
Figure 1 shows ROC curves of the submitted runs. The per shot in training data make large variations.
details of other runs are available at
http://satoh-lab.ex.nii.ac.jp/users/ledduy/Demo-MediaEval. • Annotation and ground truth are based on segments
while shots are used for experiments. Many shots are
very short, e.g. less than 1 second and might be easily
Table 2: Performance of NII’s runs(sorted by MAP) classified as non-violent shots based on the definition.
AED MAP Future work is to study how to use multiple keyframes
RunID Description Cost (%) per shot to represent violent scenes. Simple sampling
NII-run6 Global features 1.000 25.13 does not work.
Fusion of local
NII-run5 and global features 1.000 24.79
LocalFeature-VLFEAT 7. REFERENCES
NII-run1 (DSIFT/PHOW) 1.947 17.22 [1] Demarty C.H, Penet C., Gravier G. and Soleymani M.
LocalFeature-All3 The MediaEval 2011 Affect Task: Violent Scenes
NII-run4 (VLFEAT+COLORSIFT+VGG) 1.947 14.06 Detection in Hollywood Movies, MediaEval 2011
LocalFeature-COLORSIFT Workshop, September 1-2, 2011, Pisa, Italy.
NII-run3 (sift,oppsift, rgsift,csift,rgbsift) 1.975 13.12
LocalFeature-VGG(harhes,
NII-run2 harlap,haraff,heslap,hesaff) 1.976 12.87