This paper describes our experiments in Geographical Information Retrieval (GIR) in the context of our participation in the GeoCLEF 2007 Monolingual English task. Our system, called TALPGeoIR, follows a similar architecture of our previous system presented at GeoCLEF 2006 [2] with some changes in the Retrieval modes and the Geographical Knowledge Base. The system has four phases performed sequentially: i) a Linguistic and Geographical Analysis of the topics, ii) a thematic Document Retrieval search with Terrier, iii) a Geographical Document Retrieval with Geographical Knowledge Bases, iv) a Document Filtering phase. Our experiments show that Geographical Knowledge Bases can be used to improve the retrieval results of the Terrier state-of-the-art IR system by lfitering out non geographically relevant documents.
Our GIR system is a modiefid version of the system presented in GeoCLEF 2006 [
In this paper we present the overall architecture of our Geographical IR system and we describe briefly its main components. We also present the experiments, results and conclusions in the context of the GeoCLEF 2007 Monolingual English. 2 2.1
The system architecture has four phases that are performed sequentially: Topic Analysis, Textual Retrieval, Geographical Retrieval, and Document Filtering. Previously, a Collection Preprocessing phase has been applied over the textual collections. 2.2
We pre-processed the entire English collections: Glasgow Herald 1995 (GH95) and Los Angeles Times 1994 (LAT94) (i.e. 169,477 documents) with linguistic tools (described in the next subsection) to mark the part-of-speech (POS) tags, lemmas and Named Entities (NE). After this process the collection is analyzed with a Geographical Thesaurus (described in the next subsection). This information was used to built two indexes: one with the geographical information extracted from the documents (and enriched with a GKB) and another with the original textual information. We have used the Terrier Information Retrieval (IR) system to index the Textual Index.
• Geographical Index: this index contains the geographical information of the documents: the feature types appearing int the documents and the geographical information associated to each Geographical Named Entity of the document (feature type and geo-ontology path information and coordinates). Even if the place is ambiguous all the possible referents are indexed. • Textual Index: this index stores the lemmatized content of the document without added geographical information 2.3
The goal of this phase is to extract all the relevant keywords (with its analysis) from the topics. These keywords are then used by the Document Retrieval phases. The Topic Analysis phase has two main components: a Linguistic Analysis and a Geographical Analysis. 2.3.1
This process extracts lexico-semantic and syntactic information using the following set of Natural
Language Processing tools: i) TnT an statistical POS tagger [
The Geographical Analysis is applied to the Named Entities from the Title and Description and Narrative tags that have been classiefid as LOCATION or ORGANIZATION by the NERC module. This analysis uses a Geographical Knowledge Bases that has three main components: • •
Geographical Thesaurus: this component has been built joining four gazetteers that contain entries with places and their geographical class, coordinates, and other information: 1. GEOnet Names Server (GNS)2: a gazetteer covering worldwide excluding the United
States and Antarctica, with 5.3 million entries. 2. Geographic Names Information System (GNIS)3, contains 2.0 million entries about geographic features of the United States and its territories. We used a subset of 39,906 entries of the most important geographical names. 3. GeoWorldMap4 World Gazetteer: a gazetteer with approximately 40,594 entries of the most important countries, regions, and cities of the world. 4. World Gazetteer5: a gazetteer with approximately 171,021 entries of towns, administrative divisions and agglomerations with their features and current population. From this gazetteer we added only the 29,924 cities with more than 5,000 unhabitants. Each one of these gazetteers have a different set of classes. We have mapped these sets to the ADL Feature Type Thesaurus.
Feature Type Thesaurus. The feature type thesaurus of ourThe keywords used for
Geographical Thesaurus is the ADL Feature Type Thesaurus (ADLFTT). The ADL Feature
Type Thesaurus is a hierarchical set of geographical terms used to type named geographic
places in English [
[
In order to cope with ’shape lfies’ we have developped a toolbox (implemented in Prolog) allowing a simple management. The main facilities provided by the toolbox are: – Obtaining the border points of a country. – Detecting if a point belongs to a country or area. – Obtaining a polygon which encodes a certain area of a country using a 9-grid zone division (North, North-West, North-East, West, Central, East, South, South-West, Sout-East). – Getting the border points around a point P at a distance D.
– Getting near points around a point P. 2GNS. http://gnswww.nima.mil/geonames/GNS/index.jsp 3GNIS. http://geonames.usgs.gov/geonames/stategaz 4Geobytes Inc.: Geoworldmap database containing cities, regions and countries of the world with geographical coordinates. http://www.geobytes.com/.
5World Gazetteer: http://www.world-gazetteer.com
Terrier is an state-of-the-art Information Retrieval system that includes parameter-free
probabilistic retrieval approaches such as: Divergence from Randomness (DFR) models [
This module uses Terrier over a lemmatized index of the document collections and retrieves
the relevant documents using the whole content of the tags previously lemmatized.
Our Geographical Knowledge Base (described before) is used to retrieve geographically relevant
documents given the geographical terms of a Geographical IR query. The GeoKB uses a Relaxed
geographical search policy (see [
This component lfiters the documents retrieved by Terrier and the Geographical Document Retrieval module. First, the top-scored documents retrieved by Terrier that appear in the document set retrieved by the Geographical Document Retrieval module are selected. Then, if the set of selected documents is less than 1,000, the top-scored documents of Terrier that not appear in the document set of Lucene are selected with a lower priority than the previous ones. Finally, the rfist 1,000 top-scored documents are selected. On the other hand, when the system uses only Terrier for retrieval only selects the first 1,000 top-scored documents by Terrier. 2.6.1
We developed a new filtering process that using the Shape files toolbox of the GKB allows to create polygons of geographical points that enclose the geographical restriction described by the geographical terms of the topic. 3 3.1
We performed a set of experiments with the GeoCLEF 2006 topics in order to determine the top performing options for the Terrier IR platform.
The best options were a TF-IDF schema over a lemmatized collection with Porter Stemmer and Query Expansion (docs=10;terms=40). The previous configuration achieved a MAP of 0.3457 in the GeoCLEF 2006. Outperforming the BM25 and the DFR (Divergence From Randomness) schemas with MAPs of 0.3394 and 0.2862. 3.2
For the GeoCLEF 2007 evaluation we designed a set of five experiments that consist in applying, geographical knowledge filtering, Relevance Feedback, and different topic tags to an automatic state-of-the-art IR system (see Table 1). These experiments used the options that gave us the best results with the,
Basically, these experiments can be divided in two groups depending on the retrieval engines used: • Only Terrier. Two baseline experiments have been done in this group: the runs TALPGeoIRTD1 and TALPGeoIRTDN1. These runs differ uniquely in the use of the Narrative tag in the second one. Both runs use the Terrier IR system without GKBs over a lemmatized collection and applying TFIDF with Porter Stemmer and Query Expansion (docs=10;terms=40) in order to retrieve a max of 10.000 docs per topic. • Terrier & GeoKB Filtering. The runs TALPGeoIRTD2 and TALPGeoIRTDN2 use the same Terrier congfiuration than the previous runs for textual Document Retrieval and a GKB for geographical Document Retrieval. A process of Document Filtering based on a Geographical Document Retrieval re-ranks the textually retrieved docs.
The experiment TALPGeoIRTDN3 is similar to the previous experiments but uses Border Filtering and omits Query Expansion with Relevance Feedback. Due to the initial phase of our Border filtering approach, this lfitering was only applied to the topics that have a geographical relation that implies “close” or “near” and some regions. The results of the TALPGeoIR system at the GeoCLEF 2007 Monolingual English task are summarized in Table 2. This table has the following IR measures for each run: Average Precision, R-Precision, and Recall.
The runs that use Terrier and the GeoKB have a better Average Precision, R-Precision than the ones that use only Terrier. The run with the best Average Precision is TALPGeoIRTD2 with 0.2850. The best Recall measure is obtained by the run TALPGeoIRTDN1 with a 93.23% of the relevant documents retrieved. This run has the same congfiuration of the TALPGeoIRTD1 run but uses the Narrative tag. The run TALPGeoIRTDN3, that used Border Filtering without Relevance Feedback, shows an slightly improving of the MAP and Recall compared with the results of the other runs that use the Narrative tag: TALPGeoIRTDN1 and TALPGeoIRTDN2. We used Terrier, a state-of-the-art IR system for QA, to the GeoCLEF 2007 Monolingual English task. We also have experimented with an approach using both Terrier and a Geographical Knowledge Base. In this approach Terrier was used only for textual IR and the GeoKB was used to detect the geographically relevant documents. Our results show that applied GKBs can improve some retrieval results of an state-of-the-art IR system: i) the approach with Terrier and the GeoKB was slightly better in terms of MAP than the one with Terrier alone, ii) the Border Filtering approach applied without Relevance Feedback improved slightly the results in MAP and Recall.
As a future work we propose the following improvements to the system: i) the resolution of geographical ambiguity problems applying toponym resolution algorithms, ii) use Terrier with the Divergence From Randomness algorithm instead of the TFIDF, iii) the improvement and evaluation of the Shape Files toolbox and the Border Filtering algorithm.
This work has been supported by the Spanish Research Dept. (TEXT-MESS, TIN2006-15265C06-05). Daniel Ferers´ is supported by a UPC-Recerca grant from Universitat Polietc`nica de Catalunya (UPC). TALP Research Center is recognized as a Quality Research Group (2001 SGR 00254) by DURSI, the Research Department of the Catalan Government.