Robust Ad-hoc Retrieval Experiments with French and English at the University of Hildesheim

Abstract

This paper reports on experiments submitted for the robust task at CLEF 2006 ad intended to provide a baseline for other runs for the robust task. We applied a system previously tested for ad-hoc retrieval. Runs for mono-lingual English and French were submitted. Results on both training as well as test topics are reported. Only for French, positive results above 0.2 MAP were achieved.

Keywords

• Measurement
• Performance
• Experimentation

Introduction

We intended to provide a base line for the robust task at CLEF 2006. Our system applied to ad-hoc CLEF 2005 data (Hackl et al. 2005) is an adaptive fusion system based on the MIMOR model (Mandl & Womser-Hacker 2004) . For the base line experiments, we solely optimized blind relevance feedback (BRF) parameters based on a strategy developed by Carpineto et al. (Carpineto et al. 2001) . The basic retrieval engine is Lucene. Run

MAP Run Only the runs for French have reached a competitive level of above 0.2 MAP. The results for the geometric average for the English topics are worse, because low performance for several topics leads to a sharp drop in the performance according to this measure. 3

Future Work

For future experiments, we intend to exploit the knowledge on the impact of named entities on the retrieval process (Mandl & Womser-Hacker 2005) as well as selective relevance feedback strategies in order to improve robustness (Kwok 2005) .

References

Carpineto , C.; de Mori. R.; Romano. G.; Bigi . B. ( 2001 ) : An Information-Theoretic Approach to Automatic Query Expansion . In: ACM Transactions on Information Systems . 19 ( 1 ) pp. 1 - 27 .

Hackl , René; Mandl, Thomas; Womser-Hacker, Christa ( 2005 ) : Mono- and Cross-lingual Retrieval Experiments at the University of Hildesheim. In: Peters, Carol; Clough, Paul; Gonzalo, Julio; Kluck, Michael; Jones, Gareth; Magnini, Bernard (eds): Multilingual Information Access for Text. Speech and Images: Results of the Fifth CLEF Evaluation Campaign . Berlin et al.: Springer [LNCS 3491] pp. 165 - 169 .

Kwok , K.L. ( 2005 ) : An Attempt to Identify Weakest and Strongest Queries . In: ACM SIGIR 2005 Workshop: Predicting Query Difficulty - Methods and Applications . Salvador - Bahia - Brazil, August 19 , 2005 , http://www.haifa.ibm.com/sigir05-qp/papers/kwok.pdf

Mandl , Thomas; Womser-Hacker, Christa ( 2004 ) : A Framework for long-term Learning of Topical User Preferences in Information Retrieval . In: New Library World vol. 105 ( 5 /6) pp. 184 - 195 .

Mandl , Thomas; Womser-Hacker, Christa ( 2005 ): The Effect of Named Entities on Effectiveness in Cross-Language Information Retrieval Evaluation . In: Applied Computing 2005: Proc. ACM SAC Symposium on Applied Computing (SAC) . Information Access and Retrieval (IAR) Track . Santa Fe, New Mexico, USA. March 13.- 17 . 2005 . pp. 1059 - 1064 .