We describe here a methodology to combine two different techniques for Semantic Relation Extraction from texts. On the one hand, generic lexicosyntactic patterns are applied to the linguistically analyzed corpus to detect a first set of pairs of co-occurring words, possibly involved in “syntagmatic” relations. On the other hand, a statistical unsupervised association system is used to obtain a second set of pairs of “distributionally similar” terms, that appear to occur in similar contexts, thus possibly involved in “paradigmatic” relations. The approach aims at learning ontological information by filtering the candidate relations obtained through generic lexico-syntactic patterns and by labelling the anonymous relations obtained through the statistical system. The resulting set of relations can be used to enrich existing ontologies and for semantic annotation of documents or web pages.
Learning ontologies from text is a crucial task in the Semantic Web scenario: if data is semantically annotated with respect to an ontology, it can be shared between different parties on the basis of its meaning and it can be searched and retrieved in a more effective way. The state-of-the-art in ontology learning involves advanced Natural Language Processing (NLP) technologies and includes a series of tasks, starting from terminology extraction and concept definition to more complex ones such as learning taxonomic and non-taxonomic relations.
Concerning relation extraction, two main approaches can be distinguished, pattern based and distributional, both of which applied to linguistically analyzed documents.
In this paper we propose a methodology for Semantic Relation (SR) extraction from texts which combines these two complementary approaches by looking for both “syntagmatic” and “paradigmatic” relations inside textual corpora. In particular, generic “high recall but low precision” lexico-syntactic patterns are applied to the linguistically analyzed (i.e. shallow parsed) corpus to detect a first set of pairs of cooccurring words. These words, appearing close to each other inside a sentence, are involved in a “syntagmatic” relation, such as, for example, steer and car in the sentence “steer is part of the car”. On the other hand, a statistical unsupervised association system is used to obtain a second set of pairs of “distributionally similar” terms, that appear to occur in similar contexts, and possibly involved in “paradigmatic” relations (this is the case, for instance, of car and motorcycle in the sentences “I drive my car” and “Bob drives his motorcycle”.
Current research is aimed at learning ontological information by filtering the candidate relations obtained through generic lexico-syntactic patterns and by labelling the anonymous relations obtained through the statistical system. Among the possible solutions we envisage the use of candidate pairs of words inside reliable patterns (i.e. low recall but high precision patterns) to be projected onto the Web, possibly automatically learnt using machine learning technologies.
Automatic extraction of information from textual corpora for ontology learning is
now a well-known field with many different applications. Concerning SR extraction,
we may broadly classify current approaches in two groups:
Systems based on distributional properties of words. They consist in studying
cooccurrence distributions of words in order to calculate a semantic distance between
the concepts represented by those words. This distance metric can be used, for
example, for conceptual clustering ([
patterns to discover SRs between words in unrestricted texts. [
Concerning “pattern-based” approaches, several techniques aim at providing
support for the automatic (or semi-automatic) definition of the patterns to be used for
SR extraction. Marti Hearst [
[
The integration of the pattern-based and the distributional similarity approaches for the extraction of candidate semantic relations from corpora represents the main novelty of the proposed methodology.
From a semiotic point of view, it is worth distinguishing between syntagmatic and
paradigmatic SRs between words: Saussure claims that meaning arises from these two
types of relations between words. These two dimensions of meaning are often
presented as two orthogonal “axes”: syntagmatic relations involve associations
between linguistic expressions that exist “in presentia”, whereas paradigmatic
relations involve associations that exist “in absentia”. In a sentence like “the cat eats”,
for instance, the association between “cat” and “eats” is realised through their
cooccurrence within the same sentence; the semantic association between “eats” and
“sleeps”, on the other hand, exists even if it does not show up explicitly in the
sentence. To put it in other words, syntagmatic relations hold intratextually within
cooccurring words, whilst paradigmatic relations refer intertextually to words which are
absent from the text but which can be substituted one for another [
In this perspective, we can say that the distributional approach recognizes that two
words are semantically similar based on distributional similarity of the different
contexts in which the two words occur. The distributional method identifies a
somewhat loose notion of semantic similarity (sometimes addressed to as semantic
relatedness [
There have been just a few attempts to combine the two described approaches (see section 2): in this paper we propose a methodology for integrating distributional similarity with the pattern-based approach. The general idea is to first identify candidate relationships by applying the distributional approach and the general patterns on the chosen corpus, then filtering and classifying the results using reliable patterns applied to the Web.
In more details, generic “high recall but low precision” lexico-syntactic patterns (in the following, GPs, for Generic Patterns) are applied to the linguistically analyzed corpus to detect a first set of pairs of co-occurring words. These words, appearing close to each other inside a sentence, can be involved in a “syntagmatic” relation, such as the words “cougar” and “mammal” in the sentence “the cougar is a mammal”. On the other hand, a statistical unsupervised association system is used to obtain a second set of pairs of “distributionally similar” terms, that appear to occur in similar contexts, and are possibly involved in “paradigmatic” relations (e.g. “canteen” and “toilet” in the sentences “Bob goes to the canteen” and “Simon goes to the toilet”).
Once the two sets of pairs of words have been extracted, a set of reliable (“low recall but high precision”) lexical patterns are instantiated with the words and projected onto the Web, thus filtering the candidate relations obtained through GPs and labelling the anonymous relations obtained through the statistical system. The overall process can be summarized in the following sequence of steps:
a specific (raw) corpus, from which semantic relations between words are extracted; the sets of “generic” and “reliable” patterns are manually defined, each one corresponding to a specific SR; the corpus is linguistically analyzed using a battery of NLP tools; a distributionally–based algorithm for building classes of semantically– related words is used to obtain a first set of pairs denoting candidate (anonymous) relations between words; the GPs are applied to the syntactical analyzed corpus obtaining a second set of pairs denoting candidate (labelled) relationships between words; the reliable patterns are applied to the obtained sets of words, by instantiating each patterns with their morphological variations and in either order; a set of labelled word pairs representing SR instances.
reliable patterns is at the moment done by hand. The algorithm we use is inspired by
[
The relations we chose to focus on are hyponymy, meronymy, co-hyponymy and
near-synonymy1. However, we envisage the application of this methodology for the
1 We prefer to use the term (near-)synonymy since “there are very few absolute synonyms, if
they exist at all. So-called dictionaries of synonyms actually contain near-synonyms” [
A GP for hyponymy could be “X è un Y” (X is a Y) and “X di Y” (X of Y) for meronymy. It goes without saying that both exemplified patterns will produce a lot of noise in the results since they are used to express many other SR types, as noun compounds do for English. Once GPs are defined, we distinguish three kinds of reliable patterns: “close”, “partially open” and “open” patterns, depending on the way we use them to query the Web. In particular, partially open patterns (POPs, in the following) are constructed by inserting wildcards (such as “*”) inside the query and contain both words; “open patterns” (OPs) contain just one of the words and can contain wildcards; close patterns (CPs) don’t have any wildcard, contain both words and they are completely enclosed in quotation marks. Here are some examples of reliable patterns (the first three being CPs, the last being a POP) involving words A and B, one for each of the four relations of interest: • • • •
Hyponymy:
Meronymy:
Near-synonymy:
Co-Hyponymy:
“A e altri B” (“A and other B”)
“A è composto da B” (“A is made up of B”)
“A chiamato anche B” (“A also called B”)
“A e B sono” * (“A and B are” *)
Step 2: linguistic analysis of the corpus. The linguistic analysis of the corpus is
carried out by AnIta [
Identification of clusters of semantically related terms is carried out on the basis of
distributionally-based similarity measures. To this end, we used CLASS [
is mapped onto the syntactically chunked corpus. As expected, many matches are produced for each relation, the majority of which is represented by “noisy” word pairs. See 5 for some examples of obtained results.
such as “bicicletta, ruota” (bicycle, wheel), the morphological variations, in either order, are generated, such as “biciclette, ruote” (bicycles, wheels), “bicicletta, ruote”, “ruota, biciclette”, and so on. The new word pairs are then used inside the reliable patterns created in Step 1 and applied to the Web. Though we envisage the use of the Google™ API in the system implementation step, we are currently applying the reliable patterns on the Web by just querying the Google™ search engine. Every reliable pattern will be manually instantiated, applied to the Web and evaluated.
In this section we report a case study concerning some experiments we have done analysing a part of the Italian Wikipedia and looking for instances of the relations of hyponymy, meronymy, co-hyponymy and near-synonymy. The section is divided on the basis of the “five-steps” algorithm we have introduced.
Step 1: definition of generic and reliable patterns. GPs are defined just for hyponymy and meronymy, since near-synonymy and co-hyponymy are mainly identified on a paradigmatic basis in the distributional similarity step. As a matter of fact, it appears infrequent to find inside a corpus “intrasentence” co-occurrences of co-hyponyms and near-synonyms, except in rare cases, which can be found just by querying the Web as we do using the reliable patterns in step 5. For the sake of simplicity, let’s imagine to have just the two GPs introduced in section 4, one for hypernymy and one for meronymy.
Italian Wikipedia (908 articles, for a total of 788,000 words) as the corpus to be analyzed. For some details about the used NLP tools please refer to section 4.
applying the distributional system to the corpus we have obtained 4407 “semantically related” word pairs, containing correct and spurious results. Every relation instance is “anonymous”: every pair must be applied inside the reliable patterns to verify the actual presence of one of the four relations we are looking for and to provide a correct classification.
application of the two GPs has produced 553 candidate pairs for hyponymy and 19099 candidate pairs for meronymy.
detect through reliable patterns needs a slightly different strategy. CPs, for example, seem particularly suited for hyponymy, while concerning meronymy the contribution of POPs improves the results. To label word pairs obtained at step 3, all the reliable patterns must be applied, since there’s no way of knowing in advance the involved (if any) relation.
The preliminary evaluation has been conducted just for hyponymy and meronymy by manually building the reliable patterns relative to the word pairs obtained through the two GPs. Concerning hyponymy, 100 out of the 553 obtained word pairs (A, B) have been randomly selected and analysed: 66% of them were “positive” (A is a direct hyponym of B), 11% were “partially positive” (A is an indirect hyponym of B) while the remaining 24% were negative. Five reliable patterns have been applied to 10 word pairs randomly extracted from each class (positive, partially positive and negative). The results are shown in the following table: pattern 1 pattern 2 pattern 3 pattern 4 pattern 5 The first and second columns indicate the capability of each reliable pattern to recognize, respectively, direct hyponyms only and both direct and indirect hyponyms. Pattern 1, for example, recognizes exclusively direct hyponyms 88,76% of the times, direct and indirect hyponymy 97,04% of the times, but indicates a match in the remaining 2,96% of the cases when no hyponymy relation is really involved. The third column is relative to the proportion of matches produced by each pattern. It is interesting to notice a possible inverted proportionality between the harvesting value and the accuracy, though further investigations are necessary.
The following table shows the five reliable patterns we have used for hyponymy: pattern 1 pattern 2 pattern 3 pattern 4 pattern 5
A è un B - “anatra è un tipo di animale” (“duck is a type of animal”) A ed altri B - “anatra e altri animali” (“duck and other animals”) A o altri B - “anatra o altri animali (“duck or other animals”) B come A - “animali come l’anatra (“animals like the duck”)
B quali A - “animali quali l’anatra (“animals such as the duck”) Concerning meronymy, the GP produced a very noisy set of more than 19,000 word pairs. We manually selected 3630 pairs to evaluate the percentage of positive matches: 0.64% of them involved direct meronymy, 0.30% involved indirect meronymy while the remaining 99.06% were negative matches. We then extracted 5 word pairs from each class and manually applied each pair to the 10 reliable patterns we have constructed. It has been a quite long task, considering that each word pair required the manual construction (and consequent Google™ querying and validation) of 256 lexical patterns (about 25 variants for each pattern), for a total of 3840 patterns. We have ranked the word pairs on the basis of the number of matches they have obtained from the pattern application of the whole battery of patterns. The following table summarizes the results, ordered by rank: word pair ora, giorno (hour, day) cella, memoria (cell, memory) elettrone, atomo (electron, atom) rione, città (quarter, town) facciata, edificio (front, building) squama, strobilo (scale, strobile) mese, inverno (month, winter) muro, castello (wall, castle) esecuzione, musica (playing, music) osso, animale (bone, animal) azione, piede (action, foot) The last two columns indicate if the meronimy relation between the two words is present, respectively, inside Wordnet and ItalWordNet. It is interesing to notice, for example, how the word pair (cella, memoria) is not present inside both WN and IWN even if scoring an high rank. In conclusion, we have seen that reliable patterns are very selective (they exclude negative matches) and, in some cases, can even lead to reconsider some asserted ontological relations, such as about “elettrone” and “atomo”, two words not involved in a (direct) meronimical relation inside WN and IWN.
The last two examples are relative to co-hyponymy and near-synonymy: the candidate word pairs have been obtained by the distributional system. Concerning cohyponymy recognition, we envisage to adopt two strategies: i) to apply reliable “cohyponymy” patterns involving words A and B (as usual) and ii) to look for hypernyms of A and B (using open patterns, OPs) to verify if A and B share the same hypernym. For example, “adenina, timina” is a word pair obtained by the distributional system. We can apply a pattern and then analyse the results: “timina e adenina sono” (“thymine and adenine are”) → 1 match
- timina e adenina sono basi azotate... (thymine and adenine are nucleobases...) On the basis of this result, the two words are probably co-hyponyms of “base azotata” (nucleobase). To strengthen this hypothesis we can look for hypernyms of the two words (separately) by applying OPs and by comparing the resulting candidate hypernyms. Some OPs instantiated with “timina” (and “adenina”) we have tried are: “la timina è” (197 matches), “come la timina” (54 matches), “le * sono la timina” (4120 matches), “timina e altre” (1 match). From every (positive) resulting snippet we have extracted the candidate hypernyms for both words: candidate hypernyms for timina: {base azotata pirimidinica, componente, pirimidina, mutazione letifera, nucleotide, base, nome, base azotata } candidate hypernyms for adenina: {base azotata purinica, purina, sostanza, molecola organica, acido, ligando naturale, base nucleotidica, base azotata, base purinica, nucleotide, segno, componente} The intersection of the two sets is: {componente, nucleotide, base azotata}. If we combine this result with the previous one obtained with the co-hyponymy pattern we can assert that “timina” and “adenina” are co-hyponyms of “base azotata”.
Near-synonymy can be faced in a similar way. Let’s consider the word pair “puma, coguaro” (puma, cougar). Starting from near-synonymy patterns we obtain: “puma” * “chiamato anche coguaro” (“puma” * “also called cougar”) → 8 matches - Il puma (Puma concolor), chiamato anche coguaro o leone di montagna... - Il puma (Puma concolor dal 1993 [...]), chiamato anche coguaro... - vive nel continente americano [...] chiamato anche coguaro, ... I puma... OPs for hypernymy and hyponymy can be applied and the resulting sets intersected to look for near-synonymy between the two words. The idea is that if words A and B have similar hypernyms and hyponyms they can be considered near-synonyms. hypernyms for puma: {veicolo trasporto truppe, felino selvatico, telefilm, elicottero, felino, campione, veicolo, personaggio, animale selvaggio, animale} hypernyms for coguaro: {puma americano, felino, carnivoro, incrocio, animale} hyponyms for puma: {pantera della Florida, onza} hyponyms for coguaro: {pantera della Florida} Both the intersections of the two sets of hypernyms and the two sets of hyponyms are non-empty, thus reinforcing the near-synonymy hypothesis.
The paper shows a methodology to combine two different techniques for Semantic Relation Extraction from texts, namely, lexico-syntactic patterns and statistical distributional systems. The preliminary evaluation we have conducted has shown very promising results, both for hyponymy and meronymy relation extraction.
We are currently working to implement a fully automatic semantic relation extraction system based on the proposed approach: as soon as it will be available, a more accurate and articulated evaluation will be possible.