This report describes our approach to the Robust - Word Sense Disambiguation task. We applied the same index expansion technique used in 2008 for the Question Answering WSD task, with the addition of pseudo (blind) relevance feedback. In our approach, a WordNet expanded index is generated from the disambiguated document collection. This index contains synonyms, hypernyms and holonyms of the disambiguated words contained in documents. Query words are searched for in both the expanded WordNet index and the default index. The results show that the use of the extended index did not prove useful, obtaining 14 16% less in MAP with respect to the base system.
In 2008 we participated in the QA-WSD task using an index expansion method based on WordNet
hypernyms, synonyms and holonyms, which exploited the disambiguated collection [
In the following section, we describe the RobustWorSE (Robust Wordnet Search Engine) system. In section 3 we describe the characteristics of our submissions and discuss the obtained results.
The core of the system is a standard Lucene1 search engine (version 2.4.1). During the indexing
phase, we create two indices: the rst one (text ) contains all the terms of the sentence; the second
one (expanded index, or wn index) contains all the synonyms of the disambiguated words (we
consider the sense with the highest score to be the \right" sense). In the case of nouns and verbs,
it contains also their hypernyms. For nouns, the holonyms (if available) are also added to the
index, in a similar way to the GeoWorSE system that participated in the 2008 GeoCLEF track
[
The underlined words are those that have been disambiguated in the collection. For these words we can found their synonyms and related concepts in WordNet, as listed in Table 1.
Therefore, the wn index will contain the following terms: separate, part, move, position, place, labor party, political party, party, change, alter, con ict, ght, engagement, war, warfare, military action, action, reformist, argumentation, logical argument, line of reasoning, line.
During the search phase, in the default con guration, the text is searched for question terms. The top 5 resulting documents are analysed to extract up to 5 keywords that are used to expand the query. The keywords are
selected according to their tf.idf weight. Inverse document frequency is calculated over the entire document collection.
In the WSD con guration, search is carried out in a similar way, with the di erence that every noun and adjective is also searched for in the wn index.
In Table 2 we show the expansion terms obtained for the topic 147-AH : \Oil accidents and birds", using the two di erent con gurations. From the example it is possible to notice that weights of the terms from the WordNet query resulted higher than those obtained with the base query. mode No-WSD WSD term gero pigeon
i spill wildlife
spill pipeline river arco sh We submitted four runs with the WSD system, two using the NUS labeled collection and two with the UBC labeled collection. For each collection, we submitted one run using only the topic title and another one using both the title and the description. As baseline, we submitted two non-WSD runs, one in the con guration \title only" and one in the con guration \title and description".
In Table 3 we show the results obtained by the two non-WSD runs and the four WSD runs.
The results show that the use of the disambiguated collection did worsen the results obtained by the base system. There are di erences of 16% in MAP between the normal and WSD runs in the title only con guration, and up to 14:21% between in TD con guration. There is little di erence ( 1% in TD con guration) between the use of the NUS disambiguated collection and the UBC disambiguated collection. 4
The index expansion method proved to be particularly ine ective, reducing the MAP of the base system up to 16%. We still have to investigate the speci c reasons of such a negative behaviour, and the role of the pseudo relevance feedback in the obtained results.
We would like to thank the TEXT-ENTERPRISE 2.0 TIN2009-13391-C04-03 research project for partially supporting this work.