=Paper=
{{Paper
|id=Vol-1175/CLEF2009wn-VideoCLEF-PereaOrtegaEt2009
|storemode=property
|title=SINAI at VideoCLEF 2009
|pdfUrl=https://ceur-ws.org/Vol-1175/CLEF2009wn-VideoCLEF-PereaOrtegaEt2009.pdf
|volume=Vol-1175
|dblpUrl=https://dblp.org/rec/conf/clef/Perea-OrtegaMMU09
}}
==SINAI at VideoCLEF 2009==
https://ceur-ws.org/Vol-1175/CLEF2009wn-VideoCLEF-PereaOrtegaEt2009.pdf
SINAI at VideoCLEF 2009
José M. Perea-Ortega, Arturo Montejo-Ráez, M. Teresa Martı́n-Valdivia, L. Alfonso Ureña-López
SINAI research group. Computer Science Department. University of Jaén
Campus Las Lagunillas, Ed. A3, E-23071, Jaén, Spain
{jmperea,amontejo,maite,laurena}@ujaen.es
Abstract
This paper describes the second participation of the SINAI research group in the
VideoCLEF track. This year we only participated in the subject classification task. A
training collection was generated using the data provided by the VideoCLEF organi-
zation. Over this data, a supervised learning approach to classify the test videos was
conducted. We have used Support Vector Machines (SVM) as classification algorithm
and two experiments have been submitted, using the metadata files and without using
them, during the generation of the training corpus. The results obtained show the
expected increase in precision due to the use of metadata in the classification of the
test videos.
Categories and Subject Descriptors
H.3 [Information Storage and Retrieval]: H.3.1 Content Analysis and Indexing; H.3.3 Infor-
mation Search and Retrieval; H.3.4 Systems and Software
General Terms
Algorithms, Experimentation, Languages, Performance
Keywords
Image classification, Information Retrieval
1 Introduction
This paper presents the second participation of the SINAI research group at the VideoCLEF
2009 track. The goal of the track is to develop and evaluate tasks involving the analysis of
multilingual video content [6]. This year we only participate in the subject classification task. It
is about automatic tagging of videos with subject labels such as “Archeology”, “Dance”, “History”,
“Music” or “Scientific Research”. A total of 46 subject labels have been defined. The classification
process only makes use of the speech transcriptions of the videos and some metadata provided.
Our group have some experience in the field of the multimedia video retrieval [4] and image
retrieval, participating the last years in several tasks of the ImageCLEF track [3, 2, 1]. With regard
to the video categorization, we participated in VideoCLEF 2008, applying a simple approach to
resolve the classification task: to use an Information Retrieval (IR) system as classifier. The
speech transcriptions were used as textual queries and we generated a search collection based on
documents retrieved using the Google1 search engine. The results obtained showed that an IR
1 http://www.google.com/
system can perform well as video classifier if the speech transcriptions of the videos have good
quality [8].
This year we have submited some experiments following one main approach: supervised cate-
gorization using labeled samples. For that, a learning corpus has been generated using the data
provided by the VideoCLEF organization. Then, we have applied Support Vector Machines (SVM)
[5] as classification algorithm.
The following section describes how the training collection has been generated. In Section 3,
we explain briefly the use of SVM as text classifier. In Section 4, we describe the experiments and
we show the results. Finally, conclusions are presented in Section 5.
2 Generating the training corpus
The VideoCLEF 2009 training corpus consists of 262 XML files. These Automatic Speech Recog-
nition (ASR) files belong to the VideoCLEF 2008 (50 files) and TRECVid2 2007 (212 files). In
addition, there are some metadata files provided by the VideoCLEF organization. A fragment of
a ASR file and a metadata file are showed in Figure 1 and Figure 2, respectively.
[...] [...]
17785
over NOORDERLICHT (expr_id:22205)
VPRO
Dijk, Jochgem van
T00:00:06:780F1000 BG_34978-out.wmv
PT0H0M0S339N1000F Teleblik
INTERVIEWS met: prof. Robert Behringer, fysicus
Duke University, (op het terrein van een cementfabriek)
over het verrassend gedrag van korrelstructuren, hoe deze
zich manifesteren als vaste stof maar ook als vloeistof of
naar gas; de geschiedenis van het onderzoek; nieuwe onderzoeksimpulsen
door het model van Per Bak en de chaostheorie; hoe men alleen de
complexiteit begrijpt en worstelt om een goede fundamentele theorie
T00:00:07:120F1000 op te stellen; dr. Eric Clement, fysicus Universita Pierre et Marie
PT0H0M0S139N1000F Curie, toont een aantal proeven waarbij wiskundige vormen in
korrelstructuren ontstaan oa die laat zien hoe moeilijk het is
korrels van verschillende grootte met elkaar te mengen, vergelijkt
zijn proeven met die van Faraday en vertelt over de oorzaken van
de wiskundige vormen en hoe moeilijk het in de industrie is met
het korrelstructuren te werken oa het mengen bij de aanmaak van beton.
Parijs: Clement op scooter door stad, bij cementfabriek en igm
collega op werk; cementfabriek; souvenirdoosje met dwarrelsneeuw;
T00:00:07:259F1000 zandloper wordt omgedraaid en loopt, vallend zand;
PT0H0M0S110N1000F
Wetenschappelijk magazine met uiteenlopende
onderwerpen. Programma met reportages over wetenschappelijke
onderwerpen. Deze aflevering gaat over het zoeken naar
natuurkundige wetmatigheden in korrelstructuren zoals zand.
gebouw
VPRO
[...] [...]
Figure 1: Fragment of a ASR file Figure 2: Fragment of a metadata file
With regard to the ASR files, we have extracted the content of the FreeTextAnnotation la-
bels, generating a TREC file per document. Besides, we have added the content of the descrip-
tion abstract labels from the metadata files. The collection of training documents was processed
filtering the stop words and applying a stemmer. Because all the original files are in Dutch lan-
guage, we have used the Snowball stop word list for Dutch3 , which contains 101 stop words, and
the Snowball Dutch stemmer4 .
3 Using SVM as text classifier
Automatic tagging of videos with subject labels can be seen as a categorization problem, using the
speech transcriptions of the test videos like the documents to classify. One of the successful uses
of SVM algorithms is the task of text categorization into fixed number of predefined categories
based on their content. Commonly utilized representation of text documents from the field of
2 http://www-nlpir.nist.gov/projects/trecvid/
3 http://snowball.tartarus.org/algorithms/dutch/stop.txt
4 http://snowball.tartarus.org/algorithms/dutch/stemmer.html
Experiment MAP R-prec
Using metadata 0.0028 0.0089
Without using metadata 0.0023 0.0061
Table 1: SINAI results at VideoCLEF 2009
information retrieval (IR) provides a natural mapping for construction of Mercer kernels utilized
in SVM algorithms.
For the experiments and analysis carried out in this paper, the Rapid Miner5 framework was
selected. This toolkit provides several machine learning algorithms such as SVM and techniques
along with other interesting features.
4 Experiments and results
The experiments carried out in this paper are a first approximation to the automatic tagging of
videos using a text classifier. Two experiments have been submited: using the metadata files
provided by the VideoCLEF organization and without using them, during the generation of the
training corpus. The results obtained are showed in Table 1.
In order to evaluate the quality of the results, we have used two usual measures: the Mean
Average Precision (MAP) and the R-precision. Analyzing the results, we can see that the use of
metadata during the generation of the training corpus improves about 21.7% the average precision
of the classification of the test videos.
5 Conclusions
The use of metadata as an valuable source of information in text categorization has been already
applied some time ago, for example, in the categorization of full-text papers enriched by its
bibliographic records [7].
We expect to continue this work by applying a multi-label classifier, instead the multiclass
SVM algorithm used so far.
Acknowledgements
This work has been supported by the Regional Government of Andalucı́a (Spain) under excellence
project GeOasis (P08-41999), under project on Tourism (FFIEXP06-TU2301-2007/000024), the
Spanish Government under project Text-Mess TIMOM (TIN2006-15265-C06-03) and the local
project RFC/PP2008/UJA-08-16-14.
References
[1] Dı́az-Galiano, M.C., Garcı́a-Cumbreras, M.A., Martı́n-Valdivia, M.T., Montejo-Ráez, A. and
Ureña-López, L.A. Using Information Gain to Improve the ImageCLEF 2006 Collection. In
Carol Peters, Paul Clough, Fredric C. Gey, Jussi Karlgren, Bernardo Magnini, Douglas W.
Oard, Maarten de Rijke, and Maximilian Stempfhuber, editors, CLEF, volume 4730 of Lecture
Notes in Computer Science, pages 711–714. Springer, 2006.
[2] Dı́az-Galiano, M.C., Garcı́a-Cumbreras, M.A., Martı́n-Valdivia, M.T., Montejo-Ráez, A. and
Ureña-López, L.A. Integrating MeSH Ontology to Improve Medical Information Retrieval. In
Carol Peters, Valentin Jijkoun, Thomas Mandl, Henning Müller, Douglas W. Oard, Anselmo
5 Rapid Miner is available from http://rapid-i.com/
Peñas, Vivien Petras, and Diana Santos, editors, CLEF, volume 5152 of Lecture Notes in
Computer Science, pages 601–606. Springer, 2007.
[3] Dı́az-Galiano, M.C., Garcı́a-Cumbreras, M.A., Martı́n-Valdivia, M.T., Ureña-López, L.A. and
Montejo-Ráez, A. SINAI at ImageCLEFmed 2008. In Carol Peters, editor, Proceedings of the
Cross Language Evaluation Forum (CLEF 2008), 2008.
[4] Dı́az-Galiano, M.C., Perea-Ortega, J.M., Martı́n-Valdivia, M.T., Montejo-Ráez, A. and Ureña-
López, L.A. SINAI at TRECVID 2007. In Paul Over, editor, Proceedings of the TRECVID
2007 Workshop (TRECVID 2007), 2007.
[5] Joachims, T. Text categorization with support vector machines: learning with many relevant
features. In Claire Nédellec and Céline Rouveirol, editors, Proceedings of ECML-98, 10th
European Conference on Machine Learning, number 1398, pages 137–142, Chemnitz, DE,
1998. Springer Verlag, Heidelberg, DE.
[6] Larson, M., Newman, E. and Jones, G. Overview of VideoCLEF 2009: New Perspectives on
Speech-based Multimedia Content Enrichment. In Francesca Borri, Alessandro Nardi, and
Carol Peters, editors, Working Notes of CLEF 2009, September 2009.
[7] Montejo-Ráez, A., Ureña-López, L.A. and Steinberger, R. Text categorization using bibli-
ographic records: beyond document content. Sociedad Española para el Procesamiento del
Lenguaje Natural, (35), 2005.
[8] Perea-Ortega, J.M., Montejo-Ráez, A., Martı́n-Valdivia, M.T., Dı́az-Galiano, M.C. and Ureña-
López, L.A. SINAI at VideoCLEF 2008. In Carol Peters, editor, Proceedings of the Cross
Language Evaluation Forum (CLEF 2008), 2008.