Location indicators are text segments from which a geographic scope can be inferred, e.g. adjectives, demonyms (names for inhabitants of a place), geographic codes, orthographic variants, and abbreviations can be mapped to location names in one or more inferential steps. In this paper, the normalization of location indicators and treating morphology of location indicators for geographic information retrieval (GIR) within the system GIRSA (Geographic Information Retrieval by Semantic Annotation) are explored. Several retrieval experiments are performed on the German GeoCLEF 2007 data, including a baseline IR experiment on stemmed text (0.119 mean average precision, MAP). Results for this experiment are compared to results for experiments with normalized location indicators. Additionally, the latter approach was combined with an approach using semantic networks for retrieval (an extension of an experiment performed for GeoCLEF 2005). When using the topic title and description, the best performance was achieved by the combination of approaches (0.196 MAP); adding location names from the narrative part increased MAP to 0.258. Results indicate that 1) employing normalized location indicators improves MAP and increases the number of relevant documents found; 2) additional location names from the narrative increase MAP and recall, and 3) the semantic network approach has a high initial precision and even adds some relevant documents which were previously not found. For bilingual (English-German) experiments, queries were first translated into German before utilizing the translation as input to GIRSA. Performance for these experiments is generally lower, but reflect results for monolingual German. The baseline experiment (0.114 MAP) is clearly outperformed by all other experiments, achieving the best performance for a setup using title, description, and narrative (0.209 MAP).
Traditional information retrieval applies stemming to all words in a text. In the context of geographical information retrieval (GIR) on textual information, named entity recognition and classification play an important role to identify location names and to avoid stemming them. GIR is concerned with facilitating geographically-aware retrieval of information. This awareness often results from identifying proper nouns in the text, disambiguating them further into person names, organization names, and location names (geographic entities). Thus, identification of location names is typically restricted to proper nouns only.
The main goal of this paper is to investigate if one should aim at a broader GIR approach which is not solely based on proper nouns corresponding to location names. To test end, the notion of location indicators is introduced and retrieval experiments are performed by the system GIRSA (Geographic Information Retrieval by Semantic Annotation).1 The experiments are based on documents and topics for GeoCLEF 2007, the geographic information retrieval task at CLEF 2007 (Cross Language Evaluation Forum). 2 2.1
In this paper, location indicators are investigated. Location indicators are text segments from which the geographic scope of a document can be inferred. They include, but are not limited to: • Adjectives corresponding to a location.
• Demonyms, e.g. the name for inhabitants originating from a location.
• Codes for a location name, including ISO region codes, postal and zip codes.
3166-1 alpha-3); “GY” for “Guyana” (ISO 3166-1 alpha-2); “EGLL” for “Heathrow Airport, London, UK” or “LPBJ” for “Beja Air Base, Beja, Portugal” (International Civil Aviation Organization codes). • Abbreviations and acronyms for a location name, including abbreviations of adjectives.
“italienisch”/“Italian” (“Italien”/“Italy”); “Whv.” for “Wilhelmshaven”; “NRW” for “Nordrhein
• Orthographic variants, including exonyms and historic names.
• Language names in the text.
1The research described is part of the IRSAW project (Intelligent Information Retrieval on the Basis of a Semantically Annotated Web; LIS 4 – 554975(2) Hagen, BIB 48 HGfu 02-01), which is funded by the DFG (Deutsche Forschungsgemeinschaft). • Meta-information for a document, i.e. the language a document is written in.
• Unique entities associated with a geographic location, i.e. headquarters of an organization, persons, and buildings.
• The location names itself, including full names and short forms.
Typically, location indicators are not included in gazetteers, e.g. the morphology and lexical knowledge for adjectives is missing completely. Distinct location indicators contribute differently to the task of assigning a geographic scope to a document. Their importance depends on their usage and frequency in the corpus (e.g. adjectives are generally frequent) and the correctness of identifying them, because new ambiguities arise (e.g. the ISO 3166-1 code for Tuvalu (TV) is also the abbreviation for television). 2.2
The normalization of location indicators to location names takes place on different levels of linguistic analysis in GIRSA.
• Character level: In all entries of the name lexicons, diacritical marks are replaced with non-accented characters to create orthographic variants of names. These resulting orthographic variants are used as elements of a synonym set and normalized by selecting a representative for the synonym set (synset).
Example: “Que´bec” →“Quebec”. • Morphologic level: Inflectional endings for adjective and noun forms are identified and separated using a set of manually created rules and large lists of exceptions. Typical German inflectional endings of a word form (e.g. “-s” , “-es” , “-er” , “-en” , “e”) are removed before the lookup in name lexicons. (Note that location names usually do not have a plural form.) More complex cases are multi-word expressions which may contain inflectional morphology. Morphologic variations of location names are reduced to its base form.
Examples: “Berlins” →“Berlin”; “das Rote Meer” →“Rote Meer”; “des Roten Meer(e)s” →“Rote
Derivational morphology is part of connecting adjectives to location names.
Example: “bayrisch” →“Bayern”; “da¨nisch” →“Da¨nemark”. • Semantic level: Prefixes indicating compass directions are separated from the name. A database management system may view the hyphenated result as either one or two terms, depending on the search options. Thus, a search for “Norddeutschland” will also return documents containing the phrase “im Norden Deutschlands”. Also on the semantic level, a mapping between location indicators and location names takes place.
Examples: “Norddeutschland” →“Nord-Deutschland” ; “Su¨d-Frankreich” exception: “Su¨dafrika” →“Su¨dafrika”. →“Su¨d-Frankreich” ; • Lexical level: Name variations are normalized using synset representatives. The synsets contain elements referencing the same geographic location.
Example: “Burma”, “Birma” →“Myanmar”.
Of course, there is an implicit ordering of normalization steps: morphological variations are identified first, removing inflectional endings before lookup. Then, complex named entities are recognized and represented as a single term. Next, adjectives and acronyms are mapped to the expanded location name. Normalization by mapping to a synset representative is the last operation. 2.3
This year, the approach of semantic representation matching (GIR-InSicht, derived from the deep QA
system InSicht, [
For GeoCLEF 2007, query decomposition was implemented, i.e. a query can be decomposed into two dependent queries, the subquery and the main query. The subquery was answered by the QA system InSicht; the answers were integrated into the main query on the semantic network level (thereby avoiding the complicated or problematic integration on the surface level). For example, the title of topic 10.2452/57-GC “Whiskyherstellung auf den schottischen Inseln” (‘Whiskey production on the Scottish Islands’) and similarly the description of this topic lead to the subquery “Nenne schottische Inseln” (‘Name Scottish islands’). Decomposition is also applied to the alternative query semantic networks derived by inferential query expansion. In the above example, this leads to the subquery “Nenne Inseln in Schottland” (‘Name islands in Scotland’). InSicht answers the subqueries on the semantic representations of the GeoCLEF document collection and the German Wikipedia. For the above subqueries, it correctly delivered islands like “Iona” and “Islay”, which in turn lead to main query semantic networks which could be paraphrased as “Whiskyherstellung auf Iona” (‘Whiskey production on Iona’) and “Whiskyherstellung auf Islay” (‘Whiskey production on Islay’). Note that the decomposed queries are processed only as alternatives to the original query.
Another decomposition strategy produces questions aiming at meronymy knowledge based on the geographical type of a location, e.g. for a country C in the original query a subquery like “Name cities in C.” is generated, whose results are integrated into the main query semantic network. This strategy led to interesting questions like “Welcher Staat/Welche Region/Welche Stadt liegt im Himalaya?” (‘Which country/region/city is located in the Himalaya?’). In total, both decomposition strategies led to 80 different subqueries for the 25 topics. After the title and description of a topic have been processed independently, GIR-InSicht combines the results. If a document occurs in the title results and the description results, the highest score was taken for the combination.
The semantic matching approach is completely independent of the main approach in GIRSA. Some of the functionality of the main approach is also realized in the matching approach, e.g. some of the location indicators described above are also exploited in GIR-InSicht (adjectives; demonyms for regions and countries). They are not normalized, but the query semantic network is extended by many alternative semantic networks that are in part derived by symbolic inference rules using the semantic knowledge about location indicators. In contrast, the main approach exploits this information on the level of terms. 2.4
Nagel [
Buscaldi, Rosso et al. [
Wang et al. [
Previous work on GIR by members of the IICS includes experiments with documents and queries
represented as semantic networks [
The GeoCLEF 2007 documents constitute a corpus of more than 275,000 German newspaper articles from
‘Frankfurter Rundschau’, ‘Schweizerische Depeschenagentur’, and ‘Der Spiegel’ from the years 1994 and
1995 (see [
Documents are preprocessed as follows to produce different indexes:
1. S: As in traditional IR, all words in the document text (including location names) are stemmed, using
an implementation of the German snowball stemmer.
2. SL: Location indicators are identified and normalized to a base form of a location name.
3. SLD: In addition, decompounding is applied to the words in the text. German decompounding
follows the frequency-based approach described in [
The following location indicators were normalized in documents and queries for the GIR experiments: adjectives corresponding to locations, demonyms, abbreviations, orthographic variants, language names, and location names. Normalization consists of applying a set of transformation rules (covering regular variations) and looking up locations in specialized exception lists for each type of location indicator.
Basically, queries and documents are processed in the same way. The title and short description were used for creating a query. GeoCLEF topics contain a narrative part describing documents which are to be assessed as relevant. Instead of employing a large gazetteer containing location names as a knowledge base for query expansion, additional location names were extracted from the narrative part of the topic. No meronymy information is utilized for direct query expansion (because these may be just the terms the blind feedback may find and there would be a combined effect).
For the bilingual (English-German) experiments, the queries were translated using the Promt web service for machine translation.3 Query processing then follows the setup for monolingual German experiments.
The following parameter settings were used in different retrieval experiments: 1. query language: German (DE) or English (EN); 2. index type: stemming only (S), identification of locations, not stemmed (SL), decomposition of
German compounds (SLD), hybrid (SLD/O), and based on semantic networks (O); 3. query fields: combinations of title (T), description (D), and locations from narrative (N).
Parameters and results for monolingual German and bilingual English-German experiments are shown in Table 1. The table shows relevant and retrieved documents (rel ret), MAP and precision at five, ten, and twenty documents. In total, 904 documents were assessed as relevant for the 25 topics. For the run FUHtd6de, results from GIR-InSicht were merged with results from the experiment FUHtd3de in a straightforward way, using the maximum score. index S SL SLD SL SLD SLD/O O S SL SLD SL SLD fields TD TD TD TDN TDN TD TD TD TD TD TDN TDN rel ret Identifying and indexing normalized location indicators, decompounding, and adding location names from the narrative part improves performance considerably, i.e. 120 additional relevant documents are found and MAP is increased from 0.119 (FUHtd1de) to 0.258 (FUHtdn5de) in comparison to the baseline experiment.
Decompounding German nouns seems to have different effects on precision and recall (FUHtd2de vs. FUHtd3de and FUHtdn4de vs. FUHtdn5de): while more relevant documents are retrieved without decompounding, initial precision is higher when utilizing decompounding.
Topic 10.2452/55-GC contains a negation in the topic title and description ( “but not in the Alps”). However, adding the location names from the narrative part of the topic (“Scotland, Norway, Iceland”) did not notably improve precision for this topic (0.005 MAP in FUGtd3de vs. 0.013 MAP in FUHtdn5de).
A small analysis of results found by GIR-InSicht in comparison with the main GIR system reveals that for ten topics, GIR-InSicht retrieved documents, and for seven topics, it returned relevant documents (see Fig. 1 and Table 2). This approach, originating from question answering and based on a strict matching of semantic representations, returns three additional relevant documents for the combination (FUHtd6de). However, the MAP for some topics in the combined run indicates that merging by taking the maximum of two scores might be too simple. For a single topic (10.2452/52-GC), zero relevant documents were retrieved in all experiments.
Results for the bilingual (English-German) experiments are generally lower. As for German, all other experiments outperform the baseline (0.114 MAP). The best performance is achieved by an experiment using topic title, description, and location names from the narrative (0.209 MAP). In comparison with results for the monolingual German experiments, the performance drop lies between 4.2% (first experiment) and 27.1% (fifth experiment). 5
In this paper location indicators were introduced as text segments from which location names can be inferred. For the GeoCLEF 2007 experiments, different indexes containing stemmed words and location indicators normalized to location names were created. Results of the GIR experiments show that MAP is higher when using location indicators instead of location names to represent the geographic scope of a document. A broader approach to identify the geographic scope of a document is needed because proper nouns or location names do not alone imply the geographic scope of a document.
In addition, we investigated using location names extracted from the narrative part of a topic (instead of looking up additional location names in large gazetteers). The narrative contains a detailed description about which documents are to be assessed as relevant (and which not), including additional location names. Adding these location names to the query notably improves performance. This result is seemingly in contrast to some results from GeoCLEF 2006, were it was found that additional query terms (from gazetteers) degrade performance. A possible explanation is that in this experiment, only a few location names were added (3.16 location names on average for fifteen of the 25 topics with a maximum of thirteen additional location names). When using a gazetteer, one has to decide which terms are the most useful in query expansion. If this decision is based on the importance of a location, a semantic shift in the results may occur, which degrades performance. In contrast, selecting terms from the narrative part increases the chance to expand a query with relevant terms only.
The hybrid approach for GIR proved interesting, and even a few additional relevant documents were found in the combined run. As GIR-InSicht originates from a deep (read: semantic) QA approach, it returns documents with a high initial precision, which may prove useful in combination with a geographic blind feedback strategy. GIR-InSicht performs worse than the IR baseline, because only 102 documents were retrieved for ten of the 25 topics. However, more than half (56 documents) turned out to be relevant.
Several improvements are planned for GIRSA. These include using estimates for the importance (weight) of different location indicators, possibly depending on the context (e.g. “Danish coast” →“Denmark”, but “German shepherd” 6→ “Germany”), and using a part-of-speech tagger and named entity recognizer to identify location names. Finally, we plan to investigate the combination of means to increase precision (e.g. recognizing metonymic location names) with means to increase recall (e.g. recognizing and normalizing location indicators).