The task for this year is very similar to last year. However, this time we incorporate last year's experience, in particular, we explored the possibility of improving the selection of snippets, eliminating those that do not make sense, as well as those containing duplicate information. Also, it is intended to explore the real impact of the use of several languages in obtaining relevant fragments.
This year, the WebCLEF track is similar to the 2007 edition, that is: retrieving text snippets or fragments of web pages which bring up information about a topic; additionally, snippets must be in a language from a set of accepted ones. As in 2007, we have a set of topics, each with a title and a short description, as well as several documents or known sources about the topic. Additionally, for each topic, we have one or several searches in Google, with the rst 1000 documents retrieved.
The system used is basically the same as last year, for each topic we considered all documents retrieved after queries to Google as the collection of documents with which to work. These documents are to be fragmented into pieces, each of whom will be treated as a separate document.
For the queries, we use the description that we have for each topic. This query can be enriched with more terms from the known sources. So, the task can be approached like a classic problem of retrieval, and apply, consequently, conventional techniques. Organizers of task provide us the translation to plain text of documents retrieved by Google. We have valued equal all the searches in Google for same topic. UTF-8 worked ne in almost all cases, something important as there was documents in several languages and with di erent alphabets (including Cyrillic, for example). That freed us of the many problems experienced in previous editions with the detection of the coding system of each document. So, for each document translated to plain text, we have to segment it in fragments, to obtain the terms of each fragment and to calculate its weights.
To segment documents and to obtain fragments or short text passages can be applied diverse
techniques. Basically, ones are based on the size in bytes, or words; and others are oriented in
the separation in phrases or paragraphs [
A simplist approach, like the election of a orthographic character, as the period (.) like reference
to fragment the text [
Some other transformations were carried out: conversion to small letters, removing accents, removing stopwords, (with a long list of stop words for all the accepted languages), application of a simple s-stemmer.
Each fragment thus obtained and transformed was considered an independent document.
Terms were extracted and they were weighed according to scheme ATU (slope=0.2) [
From the document collection formed with snippets, we must select those that are more usefull for each topic. The key is in composing suitable queries that can produce this selection. As sources of information to compose those queries, we have topics with a short title and a brief description. Additionally, we also have, for each topic, a few documents denominated known sources, in full text.
So we can use topics (title and description) like nucleus of each query, and enrich this one with terms coming from the known sources. The known sources are complete documents, which can contain many terms. 4
Last year we worked on the formation of the queries. The basic dilemma was whether the core of each topic (title and description) could be enriched by the terms of the known sources, and to what extent the use of these terms adds useful information to the query.
Results showed little di erence in using terms of known sources or not. It's better to use
these terms, ma non tropo. However, runs of last year showed more things. Web pages are not
conventional documents; in addition to hyperlinks and hypermedia elements, they have a structure
that is not always sequentially. Many web pages are viewed by the user as a set of visual blocks
that have di erent functions and containing di erent types of information [
Unfortunately, this visual structure can not be obtained from the elements of HTML and this
a di cult area to address, although there are some jobs that are trying to solve these problems
[
We tried a very naive approach, ltering and dropping snippets based on a simple heuristics: fragments with too many blank lines, with very short lines, with a few words in relation to the size of the fragment, and so on. So, from 639 215 snippets obtained from documents , our lter deleted 165 442 (=25.88 %). This would suggest that we work with a database with a lot of noise, a deepening in the way of extracting fragments could possibly signi cantly improve results.
In similar way, last year we observed a lot of duplicated snippets. Information is replicated across the web, and so we have fragments of di erent pages that have the same information. However, as visual presentation is not always the same, the results of the conversion to plain text produces di erent strings. We used the Dice Coe cient as measure to compare snippets and discover duplicates and almost duplicates. In this case, we applied detection of duplicates on the retrieved snippets for each topic. So, if we consider snippets with Dice similarity greater than 0.7, we found that 11.08 % are duplicates ones.
On the other hand, one may wonder whether the retrieval of fragments in di erent languages provide more relevant information, and to what extent. All topics allow at least fragments into English as a useful response and, additionally, in other languages. It is expected that these fragments in other languages are derived from queries which include terms in those other languages. We made a run using queries in English only, which should allow us to compare results and assess whether the extent to which the use of other languages aid in retrieval. 5
We have described our approach to WebCLEF task, similar to last year, but incorporating the experience of the last edition. On this occasion, although on the same basis, we have focused on the e ect of ltering meaningless fragments, detect duplicates, as well as in estimating the importance of documents in languages other than English. At the time of writing these notes does not yet have the results of experiments.