Augmenting Wikidata with spatial relations specific to Geography can be useful for increasing its utility in multiple applications. In this paper, we aim to extract orientation of borders between countries in the world, from their Wikipedia text and suggest its use to augment the shares_borders_with relation in Wikidata. We propose the use of Natural Language Inference (NLI) for extracting the orientation relations from text and show that when combined with contextual lexical patterns, the performance becomes better than the standard NLI setting. spatial information extraction, orientation relation extraction, zero-shot natural language inference, orientation (c o u n t r y b o r d e r e d i n a d i r e c t i o n t o a n o t h e r c o u n t r y ) and distal (c o u n t r y h a v i n g a c e r t a i n a r e a ) facts (Examples in Table 1). The structured counterpart of Wikipedia - the Wikidata knowledge base, also captures these information pieces through properties such as shares_borders_with, located_in_the_administrative_territorial_entity and basin_country, indicating the relation between entities and corresponding values (Table 1).
Spatial information about geo-political entities such as countries, states and counties, finds mention in the first few sentences of their Wikipedia page, stressing its importance in an entity’s description. This spatial information can consist of topological (c o u n t r y l o c a t e d i n a c o n t i n e n t ), ∗Corresponding author. (G. K. Palshikar)
https://nramrakhiyani.wordpress.com (N. Ramrakhiyani)
Spatial Information from Wikipedia description Corresponding Wikidata relations
Denmark is a Nordic country in Northern Europe. (Denmark, P30:continent, Europe)
Metropolitan Denmark is the southernmost of the Scan- (Denmark, P47:shares_borders_with,
dinavian countries, lying south-west of Sweden, south Sweden), (Denmark,
P47:shares_borof Norway, and north of Germany. ders_with, Norway), (Denmark,
P47:shares_borders_with, Germany)
Spanning a total area of 42,943 km2 (16,580 sq mi),[
Multiple benchmarks [
Augmenting the Wikidata KB with missing entities, relations and values of relation qualifiers is useful for (i) increasing accessibility of structured knowledge facts both for humans and automatic agents, and (ii) generation of textual content (say a Wikipedia page) for rare entities/relations. Moreover, if the augmentation can be automated and based on robust techniques, the KB completion can be faster and accurate. The current work is a step in this direction, involving augmentation of Geography knowledge to the Wikidata KB, through an automatic unsupervised text mining approach.
The overall problem is to supplement the shares_borders_with property between countries with a qualifier direction_relative_to_location wherever it is not added to the property. The qualifier, particular to this property, describes the orientation of the border with respect to the main entity. For example, in the Wikidata entry for France1, it is shown to share land borders with 10 countries, but the qualifier is available only for one case (‘north’ with Belgium). However, in the Wikipedia page for France, the sentence - I t s l a n d b o r d e r s c o n s i s t o f B e l g i u m a n d L u x e m b o u r g i n t h e n o r t h e a s t , G e r m a n y a n d S w i t z e r l a n d i n t h e e a s t , I t a l y 1https://www.wikidata.org/wiki/Q142 a n d M o n a c o i n t h e s o u t h e a s t , a n d A n d o r r a a n d S p a i n i n t h e s o u t h a n d s o u t h w e s t , clearly indicates the other border orientations, not captured in France’s Wikidata KB relations.
At a finer level, the problem, given the above sentence, is to extract the direction relations between the described subject entity, (also known as Trajector in Spatial Information Extraction literature) with other entities (Landmark) mentioned in the sentence. So in this example, the goal is to extract relations such as [Its (France), northest, Luxembourg], [Its (France), east, Germany] and [Its (France), south, Spain]. Once these relations are extracted, the augmentation of the corresponding shares_borders_with properties for France can be supplemented with the correct direction_relative_to_location qualifiers. This process can be carried out similarly for all countries and their corresponding property
The standard approaches in Spatial Information Extraction are primarily supervised and work
for specific tasks. One of our earlier work [
PLM based NLI requires the text under consideration to be posed as a premise which is coupled with a suitable hypothesis for a textual entailment task. The task involves classifying the premisehypothesis pair into whether the hypothesis logically follows from the premise (Entailment) or contradicts it (Contradiction) or isn’t related to it (Neutral). The hypothesis allows the inclusion of the class information in the premise-hypothesis pair and hence, devising the hypothesis becomes an important part of the exercise. A hypothesis comprises of two customizable parts - a template and a class-indicating phrase. In this paper, we use the direction names as the class-indicating phrases. The template can be devised using two approaches. 2Code for both these recent approaches is unavailable and hence both require a separate significant reproducibility efort, which has been kept as later work.
As a basic hypothesis template, we propose the use of the arrangement “Trajector shares borders with Landmark to the Direction”. So given the input text D e n m a r k i s l y i n g s o u t h w e s t o f S w e d e n , s o u t h o f N o r w a y , a n d n o r t h o f G e r m a n y . and one of the directions/classes (say north), the premise-hypothesis pair will be (P: D e n m a r k i s l y i n g s o u t h w e s t o f S w e d e n , s o u t h o f N o r w a y , a n d n o r t h o f G e r m a n y . , H: D e n m a r k s h a r e s b o r d e r s w i t h G e r m a n y t o t h e n o r t h . ). Such pairs can be created for all directions and hence, for each Trajector and Landmark entity pair, 8 such premise-hypothesis pairs will get created.
A detailed observation of diferent sentences reveals that there is a specific way in which the borders relation is mentioned in a sentence. For example, in the Denmark example, . . . i s l y i n g t o t h e n o r t h o f . . . is used to express the borders relation, which is diferent from Afghanistan, where . . . i s b o r d e r e d b y . . . t o t h e e a s t is used. We identified about 10 diferent higher level mention styles, a subset of which is shown in Table 2 in the form of lexical patterns. To aid the PLM in performing an informed entailment, we propose to adapt the hypothesis template based on the pattern used in the premise. So in this example, the hypothesis for Denmark will be constructed using the “lying” pattern and for Afghanistan will use the “bordered by” pattern. We hypothesize that combining the power of classical lexical patterns with PLM’s attention mechanism and pre-training can lead to better results than using basic/standard hypothesis templates as above.
In the zero-shot setting, a PLM pretrained for the NLI task, is then fed each of these instantiated premise-hypothesis pairs and the PLM predicts whether there is an Entailment, Contradiction or Neutral between them. The premise-hypothesis instance for which the PLM predicts an Entailment with the highest confidence is selected and the hypothesis’ corresponding direction is predicted as the final class for the premise/input text.
As the baseline, we consider a prompt based mask filling approach. We use the hypothesis
sentences, created as part of the proposed approach (Section 3.1.2), as the prompts where in
place of the Direction we keep the [MASK] token. We employ PLMs which are trained for a
Masked Language Modelling (MLM) task, and get them to fill the correct direction in place of the
MASK token. To provide the premise-like support, we devise another baseline on lines similar
to the prompting with demonstrations idea proposed in Gao et al. [
As the first step in the experimentation, we construct a dataset of sentences which emit direction information. Based on these sentences we create the premise hypothesis pairs to be consumed by the NLI approach and the masked prompts for the baseline.
• We first use a SPARQL query on Wikidata to find the list of countries which have a shares_borders_with property. This led to creation of a list with 177 countries. • We use the MediaWiki API3 to obtain the first 10 sentences of the Wikipedia page of each of the countries in the list. In all, the total number of sentences obtained were 1766. • We then devise a simple rule to retain any sentences which have atleast one direction and two country names present and filter out the rest of the sentences from each entity’s sentences. We further filtered sentences, which didn’t communicate a bordering relation.
After this filtering, the total number of sentences left was 143. • It is important to note that, this filtering leaves us with sentences which refer to the Trajector/main entity with pronouns such as I t or common nouns such as T h e c o u n t r y , T h e n a t i o n and T h e a r c h i p e l a g o . For creating proper premises and prompts, we replaced these mentions in the sentences with the proper Trajector entity names.
For the creation of the gold standard dataset, all bordering relations in the 143 sentences were labelled. Each gold relation is of the form (Trajector, Direction, Landmark) and to be read is The Trajector entity shares border with the Landmark entity to the Direction. So for example, a gold relation (Denmark, south, Germany) is understood as Denmark shares border with Germany to the south. A total of 562 gold relations were labelled in this manner.
For the NLI approaches, we experiment with the bart-large-mnli4 and roberta-large-mnli5
models from the huggingface library. Both these models are standard transformer models, tuned
further on the Multi-NLI dataset [
In Table 3, we report the results for the NLI approach (basic and patterns template) and the prompting baselines, in terms of Precision@k (k = 1 and 3) which checks if the correct direction is predicted at the kℎ rank in the predictions.
As can be observed in Table 3, the NLI approach is significantly better than the prompting baselines irrespective of the models. Also as hypothesized, adapting the NLI hypotheses with lexical patterns helps in improving the performance by 4 to 8%. With respect to the NLI PLMs, the roberta-large-mnli model performs slightly better than the bart-large-mnli model.
Some peculiar dificulties observed in the task are: (i) Some countries are referred to diferently in diferent contexts and hence the hypothesis creation is missed for non-listed mentions. For example, the Wikidata entry titled People’s Republic of China is the oficial reference of China. However, it is referred to simply as China in descriptions of its neighbours such as India. (ii) Some countries have names which are subsets of other countries and hence, unwanted mentions of those countries get considered during hypothesis creation. For example, The Democratic Republic of Congo and the Republic of Congo; or Sudan and South Sudan. We also investigated why prompting based approaches are performing so poorly, in spite of already devised using the lexical patterns. Firstly, the token predicted at the MASK token is many times not even a direction mention. For example, D e n m a r k i s l y i n g [ M A S K = w i t h i n ] o f N o r w a y . Secondly, single word directions such as north, east are predicted more than the bi-word directions such as northeast, leading to incorrect predictions for such bi-word orientations.
We aim at adding orientation information to Wikidata KB’s shares_borders_with property between countries of the world. We demonstrated the use of Natural Language Inference (NLI) based querying of PLMs for this relation extraction task. We also showed that adapting the NLI hypotheses based on the mention of the property in the input text, boosts the performance further. Overall, we envisage that using NLI based techniques can be a promising direction for spatial information extraction, particularly relating to geographical and geo-political entities.