In this paper, we run our content-based video genre classication system on the MediaEval evaluation corpus. Our system is based on several low level audio-visual cues, as well as cognitive and structural information. The purpose of this evaluation is to assess our content-based system's performance on the diversi ed content of the blip.tv web-video corpus, which is described in detail in [5].
Automatic genre classi cation is an important task in
multimedia indexing. Several studies have been conducted on
this topic. A comprehensive overview of these studies on
TV genre classi cation can be found in [
Cognitive and structural features are proposed in [
To bene t from the audio information of each clip, we compute four features from the audio signal. All features are extracted from mono-channel audio with 16 kHz sample rate and a 256 kbit/s bit rate. The features include MFCC, Zero Crossing Rate and Signal Energy, and are utilized using di erent representations. 2.3
We used six di erent low level visual features which represent color and texture information in the video. 2.3.1
Histogram: We use the HSV color space and build a
histogram with 162 bins [
Color moments: We use a grid size of 5 5. The rst three
order color moments were calculated in each local block in
the image and the Lab color space is used [
Autocorrelogram: Autocorrelogram captures the spatial
correlation between identical colors. 64 quantized color bins
and ve distances are used [
Co-occurrence texture: As proposed in [
Wavelet texture grid: We calculate the variances of the
high-frequency sub-bands of the wavelet transform of each
grid region. We performed 4-level analysis on a grid that
has 4 4 = 16 regions. Haar wavelet is employed, as in [
Edge histogram: For the edge histogram, 5 lters as proposed in the MPEG-7 standard are used to extract the kind of edge in each region of 2 2 pixels. Then, those small regions are grouped in a certain number of areas (4 rows 4 columns in our case) and the number of edges matched by each lter (vertical, horizontal, diagonal 45 , diagonal 135 and non-directional) are counted in the region's histogram. 3.
Classi cation is performed using multiple SVM classi ers. As can be seen in Fig. 1, content-based features are extracted from each video and are used as input for separate MFCC SVM Model
Wavelet SVM Model
Wavelet SVM Model ∑ ∑
Majority Voting SVMs, one for each genre and feature. Classi cation output of each SVM is summed up over all features for each genre and a genre is picked via majority voting.
The evaluation of this years MediaEval genre tagging task was performed on 1727 clips form blip.tv, distributed unevenly over 26 categories including a default category. Single label classi cation is performed and mean average precision (MAP) is used as the o cial performance measure. Training of the SVMs was conducted on approximately 100 videos for each genre, except for autos and vehicles where only 14 clips were available. These training videos are from a larger additional set of blip.tv videos. However, since we had limited time, it was not possible to process all these videos. Therefore, we limited the number of training videos per genre to 100 videos, which are randomly selected for each genre. Because our system works as a single label classi cation system, we also computed simple classi cation accuracy and calculated a 2nd MAP performance with a similarity value of 1, instead of a very low probability output of our system.
All in all, 5 runs were evaluated using these three evaluation measures. In our case, a combination of all feature sets (run1) and each feature category like visual (run2), aural (run3), cognitive (run4) and structural (run5) are evaluated independently. The results are presented in Table 1. The least contribution comes from the cognitive features, while the visual features (run 2) contribute the most to the overall performance, outperforming the other runs in the MAP performance measures and achieving almost the same classi cation accuracy as all feature sets together. From the six available visual features color moments and wavelet texture show the best classi cation results with 20% and 23%, respectively.
The best results (greater 50%) were achieved in the web development (66.6%), mainstream media (68.9%), food anddrink (61.1%), movies and television (58.5%) and literature category with 89.6%. Worst results (under 10%) showed documentary (4.5%), educational (3.2%), health (9.5%), travel (7.1%) and videoblogging with 0%.
MAP 2nd MAP Accuracy (%) run1 0.0023 0.0038 28.2 run2 0.0035 0.006 27.5 run3 0.001 0.001 13.9 run4 0.001 0.0012 1.3 run5 0.003 0.0028 5.4
Our experiments show that a content-based system which
is able to achieve nearly perfect accuracy on TV datasets
(95% and 99%, see [
More interestingly the low-level visual and aural features show more promising results than the selected higher-level cognitive and structural cues. Either it is not possible to cover the variety, or overall resemblance of all videos with these features or more promising high-level features have to be found by analyzing the properties of the web-videos.
Because of the limits of content-based systems in this area, the usage of metadata and other sources like ASR engines is desirable to be able to attain a robust genre classi cation system.
This study is funded by OSEO, French State agency for innovation, as part of the Quaero Programme. 5.