The interest of users for geographically constrained information in the Web has increased over the past years, boosted by the availability of services such as Google Maps 1 . Sanderson and Kohler [1] showed that 18.6% of the queries submitted to the Excite search engine contained at least a geographic term, while Gan et al. [2] estimated that 12.94% of queries submitted to the AOL search engine expressed a geographically constrained information need. Most of the geographical information contained in the Web and unstructured text is composed by toponyms, or place names. There are two main problems that derive from using toponyms to represent geographical information. The first one is the polysemy of toponyms, or toponym ambiguity: a toponym may be used to represent more than one place, such as "Puebla" which may be used to indicate the city at 19 o 3 N, 98 o 12 W, the state in which it is contained, a suburb of Mexicali in the state of Baja California, or three more small towns in Mexico. The second problem is that the mere inclusion of a toponym in a document does not always mean that the document is geographically relevant with respect to the region or
We would like to thank the TIN2009-13391-C04-03 research project for partially supporting this work. 1 http://maps.google.com area represented by the toponym. In the first case, the solution is constituted by the Toponym Disambiguation (TD) task, also named toponym grounding or resolution; in the second case, the solution is to carry out Geographic Scope Resolution, which is also affected by the problem of toponym ambiguity [3].
The Geonames ontology2 provide users with RDF description of more than 6 million places. The use of this ontology would allow to include geospatial semantic information in the Web, eliminating the need of toponym disambiguation. Unfortunately, as noted by [4], in the Web "references to geographical locations remain unstructured and typically implicit in nature", determining a "lack of explicit spatial knowledge within the Web" which "makes it difficult to service user needs for location-specific information". In this paper, with the help of the Geooreka!3 system [5], a prototype web search engine developed at the Universidad Politécnica of Valencia in Spain, we will the problems that users interested in geographically constrained information may found because of the ambiguity of toponyms in the web.
Geooreka! is a search engine developed on the basis of our experiences at Geo-CLEF4 [6,7], which suggested us that the use of term-based queries could not be the optimal method to express a geographically constrained information need. For instance, it is common for users to employ vernacular names that have vague spatial extent and which do not correspond to the official administrative place name terminology. Another issue is the use of vague geographical constraints that are difficult to automatically translate from the natural language to a precise query. For instance, the query "Cultivos de tabaco al este de Puebla" ("Tobacco plantations East of Puebla") presents a double problem because of the ambiguity of the place name and the fact that the geographical constraint "East of" is vague (for instance, it does not specify if the search should be constrained within Mexico or extend to other countries).
These issues are addressed in Geooreka! by allowing the user to specify his geographical information needs using a map-based interface. The user writes a natural language query in order to represent the query theme (e.g., "Cultivos de tabaco") and selects a rectangular map in a box (Figure 1), representing the query geographical footprint. All toponyms in the box are retrieved using a PostGIS database, and then the Web is queried in order to check the maximum Mutual Information (MI) between the thematic part of the query and all the places retrieved. The complete architecture of the system can be observed in Figure 2. Web counts and MI are used in order to determine which combinations theme-toponym are most relevant with respect to the information need expressed by the user (Selection of Relevant Queries). In order to speed-up the process, web counts are calculated using the static Google 1T Web database5 , indexed using the jWeb1T interface [8], whereas Yahoo! Search is used to retrieve the results of the queries composed by the combination of a theme and a toponym.
The key issue in the selection of the relevant queries is to obtain a relevance model that is able to select pairs theme-toponym that are most promising to satisfy the user's information need. On the basis of the theory of probability, we assume that the two component parts of a query, theme T and a place G, are independent if their conditional probabilities are independent, i.e., p(T |G) = p(T ) and p(G|T ) = p(G), or, equivalently, their joint probability is the product of their probabilities:
If probabilities are calculated using page counts, that is, as the number of pages in which the term (or phrase) representing the theme or toponym appears, divided by F max = 2, 147, 436, 244 which is the maximum term frequency contained in the Google Web 1T database, then p(T ∩ G) is the expected probability of co-occurrence of T and G in the same web page. It is clear that this represents a rough estimation of the fact that T occurred in G, since the mere inclusion of G in a page where T is mentioned does not guarantee the semantic relation between G and T .
Considering this model for the independence of theme and place, we can measure the divergence of the expected probability p(T ∩ G) from the observed probability p(T ∩ G): the more the divergence, the more informative is the result Fig. 2. Architecture of Geooreka! of the query. The Kullback-Leibler measure [9] is commonly used in order to determine the divergence of two probability distributions.
This formula is exactly one of the formulations of the Mutual Information (MI) of T and G, usually denoted as (I(T ; G)).
Geooreka! has been evaluated over the GeoCLEF 2005 test set, in order to compare the results that could be obtained by specifying the geographic footprint by means of keywords and those that could be obtained using a map-based interface to define the geographic footprint of the query. With this setup, topic title only was used as input for the Geooreka! thematic part, while the area corresponding to the geographic scope of the topic was manually selected. Probabilities were calculated using the number of occurrences in the GeoCLEF collection. Occurrences for toponyms were calculated by taking into account only the geo index. The results were calculated over the 25 topics of GeoCLEF-2005, minus the queries in which the geographic footprint was composed of disjoint areas (for instance, "Europe" and "USA" or "California" and "Australia"), which could not be processed by Geooreka!. Mean Reciprocal Rank (MRR) was used as a measure of accuracy. The GIR system GeoWorSE, where queries are specified by text, was used as a baseline [10]. Table 1 displays the obtained results. The results show that the web-based results are sensibly worse than those obtained on the static collection. This is due primarily to two reasons: in the first place, because topics were tailored on the GeoCLEF collection. Therefore, some topics refer explicitly to events that are particularly relevant in the collection and are easier to retrieve. For instance, query GC-005 "Japanese Rice Imports" targets documents regarding the opening of the Japanese rice market for the first time to other countries; "Japan" and "Rice" in the document collection appear together only in such documents, therefore it is easier to retrieve the relevant documents when searching the GeoCLEF collection.
The second factor affecting the results for the Web-based system is ambiguity of toponyms, which does not allow to correctly estimate the probabilities for places. For instance, in the results obtained for topic GC-008 ("Milk Consumption in Europe"), the MI obtained for "Turkey" was abnormally high with respect to the expected value for this country. The reason is that in most documents, the name "turkey" was referring to the animal and not to the country. This kind of ambiguity represents one of the most important issue at the time of estimating the probability of occurrence of places. Ambiguity (or, better, the polysemy of toponyms) grows together with the size and the scope of the collection being searched. The GeoCLEF collection was also semantically tagged using WordNet and Geonames IDs to identify the places referenced by toponyms, while Web content is rarely tagged using precise IDs, therefore increasing the chance of error in the estimation of probabilities for places which share the same name.
There are three kind of toponym ambiguity that can be recognised (after the two main types identified by [11]:
-Geo / Non-Geo ambiguity: in this case, a toponym is ambiguous with respect to another class of name (such as "Turkey" which may be the animal or the country); -Geo / Geo ambiguity of different class: for instance, "Puebla" the city or the state; -Same class Geo / Geo ambiguity.
The solution in all cases would be to use an ontology to precisely identify places in documents; the only difference is the amount of information that the ontology should include. For the first type of ambiguity, the only information needed is whether the name represents a place or not. In the second case, we would also need to know the class of the place. Finally, in the Geo / Geo ambiguity, we may differentiates places using their coordinates or by knowing the including entity, or both. The Geonames ontology contains all these information and represents the best option at the time of geographically tag place names.
The results obtained with Geooreka! over a static, semantically-labelled (at least from a geographical viewpoint) collection compared to the results obtained in the Web showed that the imprecise identification of places is a problem for search engines destined to users who are interested in searching for geographically constrained information. The use of precise semantically tagging schemes for toponyms, such as Geonames RDF, would allow these search engines to produce more reliable results. Spreading the use of geographical tagging for the Semantic Web would also allow users to mine information using geographical constraints in a more effective way. In this sense, we would like to encourage the use of Geonamen in order to produce accurate geographically tagged Web content.