SINAI at INFILE 2009: Experiments with Google News

Abstract

This paper describes the SINAI team participation in the INFILE routing and ltering track of the CLEF campaign. This is the rst participation of the SINAI research group in the INFILE task. We have participated in the batch ltering subtask and submitted two experiments: one using the topics' text as learning data to train a classi er, and another one where training data has been constructed from Google News pages. Our results show that our use of Google News did not improved the classi cation obtained using only topics description.

Keywords

• Text classi cation
• Information Retrieval

Introduction

only topics texts were used a learning data. The learning algorithm was Support Vector Machines [ 5 ] on both cases. 2

Experiment Description

A traditional supervised-learning scheme has been followed. The di erence between the runs submitted rely on the training corpus used. One was on Google News entries and another on the descriptions of the topics provided.

The Google News corpus was generated by querying Google News on each of the topics keywords. For example, topic 101 contains the keywords doping, legislation doping, athletes, doping substances and ght against doping. For each of these keywords, 50 links were retrieved, downloaded and their HTML cleaned out. In this way, about 200 documents existed per topic.

Once each corpus was generated, a SVM model was trained on it. This is a binary classi er turned into a multi-class classi er by training a di erent SVM model per topic. The topic with the highest con dence was selected as label for the incoming document and, therefore, the document was routed to that topic. It is important to note here that a label was proposed for all of the incoming documents, that is, no document was left without one of the 50 labels (topics). 3

Results and Discussion

Overall results are displayed in Table 1. The results obtained are discouraging: few relevant assignations are made. In fact, the use of Google News as learning source leads to very poor results.

Topics descriptions Retrieved Relevant Relevant Retrieved

Conclusions and Further Work

Google News as a source of information for generating learning corpus has shown quite bad results. After inspecting these problematic corpus, we found that huge amount of useless text was not ltered. Therefore, we plan to improve the quality of the data extracted from the web in order to avoid undesirable side e ects due to noisy content.

Although the results obtained in this task are really very low in terms of performance, it represents a challenge in text mining, as real data has been used, compared to previous too controlled corpora. We expect to continue our research on this data, and analyze in depth the e ect of incorporating web content in ltering tasks. 5

Acknowledgements

This work has been supported by the Andalusian Regional Government (Spain) under excellence project GeOasis (P08-41999), under project on Tourism (FFIEXP06-TU2301-2007/000024), the Spanish Government under project Text-Mess TIMOM (TIN2006-15265-C06-03) and the local project RFC/PP2008/UJA-08-16-14.

References

[1] Romaric Besan A ~xon, Stephane Chaudiron, Djamel Mostefa, Ismail Timimi, and Khalid Choukri . The in le project: a crosslingual ltering systems evaluation campaign . In Proceedings of the Sixth International Language Resources and Evaluation (LREC'08) , Marrakech, Morocco, may 2008 . European Language Resources Association (ELRA) . http://www.lrecconf.org/proceedings/lrec2008/.

[2] Francisco M. Couto , Bruno Martins, and Mario J. Silva . Classifying biological articles using web resources . In SAC '04: Proceedings of the 2004 ACM symposium on Applied computing , pages 111 { 115 , New York, NY, USA, 2004 . ACM.

[3] M.C. D az- Galiano , J.M. Perea-Ortega , M.T. Mart n-Valdivia, A. Montejo-Raez , and L.A. Ure na Lopez. Sinai at trecvid 2007 . In Paul Over, editor, Proceedings of the TREC Video Retrieval Evaluation 2007 (TRECVID'07) , 2007 .

[4] Risto Gligorov , Warner ten Kate, Zharko Aleksovski, and Frank van Harmelen. Using google distance to weight approximate ontology matches . In WWW '07: Proceedings of the 16th international conference on World Wide Web , pages 767 { 776 , New York, NY, USA, 2007 . ACM.

[5] Thorsten Joachims . Text categorization with support vector machines: learning with many relevant features . In Claire Nedellec and Celine Rouveirol , editors, Proceedings of ECML-98, 10th European Conference on Machine Learning, number 1398 , pages 137 { 142 , Chemnitz , DE, 1998 . Springer Verlag, Heidelberg, DE.