This year, the SICS team decided to concentrate on query processing and on the internal topical structure of the query: we have identified this as one of the major bottlenecks for cross-lingual access systems. Previous years, the SICS team has investigated, among other issues, how to translate compounds [2]. Compound translation is non-trivial due to dependencies between compound elements and has been treated in various ways in the treatment of compounding languages such as Swedish [1, 3, e.g.]. We decided this year to investigate the topical dependencies between query terms, under the hypothesis that the complexity of translating compounds is a special case of the more general case of understanding the respective topicality of query terms.

The question under investigation is how much each query term contributes in terms of topicality in the documents of the collection under consideration. If a query term happens to be non-topical or noise, it should be discarded or given a low weight when ranking retrieved documents; if a query term shows high topicality its weight should be boosted. Our base system is used with two different enhancements to test the hypothesis that boosting topically active terms is beneficial for retreival results.

The French target collection and the French topics were lemmatized and normalized using the commercially available FDG tools from Connexor Oy, described in several publications [8, e.g.].

The text retrieval engine used for our experiments is based on a standard retrieval system being developed at SICS. The system is described in more detail in our CLEF paper from 2002 [7]. The French target collection was indexed by the system and the translated French queries were used to retrieve texts from the French collection without manual intervention.

In retrieval, query terms were weighted by standard tf.idf metrics, and, in contrast with earlier years, documents with several of the query terms were given a boost higher than documents with the equivalent number of occurrences but distributed over fewer different query terms.

In this experiment, we use distributional information to weight words selected from the query description field. The idea is to select words with similar distributional properties, since they are assumed to indicate similar topics. As an example, consider query number 251, where supposedly the term "médecine" is a good descriptor. We would then want to boost the weight of query words that are topically similar to "médecine" but that occur in other documents (it would be no point in selecting words that occur in exactly the same documents, since we retrieve those documents anyway by using the term "médecine"). Considering the example query, we would supposedly like to include words such as "homéopathie", "chiropractie", "acupuncture", and "thérapie". Our hypothesis is that we can use second order co-occurrence information to find such query words.

Our approach is based on Random Key Indexing [4,5], a technique for the efficient and tractable analysis of cooccurrence statistics. Random Key Indexing incrementally collects distributional data for terms in the text collection under consideration and can be used to build a vector space based on those data. In this experiment we use Random Key Indexing1 to collect second order co-occurrences to accumulate a word space in which words with similar distributional properties are located close to each other. We compute distributional similarity between words using the cosine of the angles between "context vectors" that represent their distributional profiles. The cosine values are then used to weight the words in the query description field.

Note that we use second order co-occurrences, since we want to find words with similar distributional statistics that do not occur in the same documents, but that occur in the same type of contexts. Using first order co-ocurrences (as is done in, e.g., Latent Semantic Analysis) would merely find words that occur in similar documents, which is not beneficial for the adhoc Information Retrieval task, since those documents are found by the system by default.

Using an analysis of query term distribution in the target collection, Katz' γ is calculated for each term in the query. This can be understood as the estimated probability for the term to appear at least twice in any given text and is calculated by as the relative frequency of texts with at least two occurrences of the term under consideration to texts with only one occurrence of it. The intuition underlying Katz' γ is that singleton occurrences may be happenstance noise whereas repeated occurrences of a term are likely to be topical [6]; the intuition behind our use of the measure is that terms that often are likely to be topical are likely to be of more interest as regards query relevance than terms that often occur non-topically.

The first submission (v) used the baseline system without modification. The second submission (b) boosted query terms according to their location in the vector space as provided by random key indexing by multiplying the standard tf.idf score with the cosine between it and the closest neighbor of the other query terms. The third submission (k) boosted terms that are likely to be topical by multiplying the standard tf.idf score with its γ. The results were reasonably good with half of the fifty queries on or above median. The two boosting schemes proved to deliver improved results.

The results of the boosting schemes delivered uncontroversially improved results. One scheme examined the individual character of the terms; the other the relation between query terms. These are two different avenues of analysis and will most likely provide different (and even better) results if pursued further. These results will also provide impetus for the further study of translation of complex terms -the question which first prompted this set of experiments in the first place.