Here is discussed how to construct domain ontologies with both taxonomic and non-taxonomic conceptual relationships, exploiting a machinereadable dictionary (MRD) and domain-specific texts. The taxonomic relationships come from WordNet in the interaction with a domain expert, using the following two strategies: match result analysis and trimmed result analysis. The non-taxonomic relationships come from domainspecific texts with the analysis of lexical cooccurrence statistics; based on WordSpace to represent lexical items according to how semantically close they are to one another. We have done case studies in the field of law. The empirical results show us that our environment can support a user in constructing domain ontologies.
Although ontologies have been so popular in many application areas, we still face the problem that it takes many costs to build up them by hand. In particular, since domain ontologies have the senses specific to application domains, human experts have to make huge efforts with constructing them entirely by hand.
In order to reduce the costs, automatic or semi-automatic
methods have been proposed using knowledge
engineering techniques and natural language processing ones
As domain ontologies have been applied to widespread
areas, such as knowledge sharing, knowledge reuse, software
agents and information integration, we need software
environments that support a human expert in constructing the
domain ontologies with not only taxonomic conceptual
relationships but also non-taxonomic ones. In order to develop the
environments, it seems to be better that we put together two or
more techniques such a knowledge engineering, natural
language processing, machine learning and data engineering, as
seen in the workshop on ontology learning in ECAI2000
Here in this paper, we extend DODDLE into DODDLE II that acquires both taxonomic and non-taxonomic conceptual relationships, exploiting WordNet[Fellbaum 1998] and domain-specific texts with the automatic analysis of lexical co-occurrence statistics, based on WordSpace that has the idea that a pair of terms with high frequency on co-occurrence statistics can have non-taxonomic conceptual relationships. Furthermore, we evaluate how DODDLE II works in the field of law, the Contracts for the International Sale of Goods (CISG). The empirical results show us that DODDLE II can support a law expert in constructing domain ontologies. 2
Figure 1 shows an overview of DODDLE II: a domain ontology rapid development environment that has the following two components: taxonomic relationship acquisition module using WordNet and non-taxonomic relationship learning module using domain-specific texts. A set of domain terms is given to DODDLE II.
The taxonomic relationship acquisition module does spell match between the input domain terms and WordNet. The spell match links these terms to WordNet. Thus the initial model from the spell match results is a hierarchically structured set of all the nodes on the path from these terms to the root of WordNet. However the initial model has unnecessary internal terms (nodes) not to contribute to keeping topological relationships among matched nodes, such as parent-child relationship and sibling relationship. So we can trim the unnecessary internal nodes from the initial model into a trimmed model, as shown in Figure 2 process. In order to refine the trimmed model, we have the following two strategies in the interaction with a user: match result analysis and trimmed result analysis that will be described later.
The non-taxonomic relationship learning module extracts the pairs of terms that should be related by some relationship from domain-specific texts, analyzing lexical co-occurrence statistics, based on WordSpace that is a multi-dimensional, real-valued vector space where the cosine of the angle between two vectors is a continuous measure of their semantic relatedness. Thus the pairs of terms extracted from domainspecific texts are the candidates for non-taxonomic relationships. Thus putting together taxonomic and non-taxonomic relationships, we can get concept specification templates for the input domain terms, although the relationships should be identified in the interaction with a human expert. 3
After getting the trimmed model, TRA module refines it in the interaction with a domain expert, using the following two strategies: match result analysis and trimmed result analysis.
Looking at the trimmed model, it turns out that it is divided into a PAB (a PAth including only Best spell-matched nodes) and a STM (a Sub-Tree that includes best spell-matched nodes and other nodes and so should be Moved) based on the distribution of best-matched nodes. On one hand, a PAB is a path that includes only best-matched nodes that have the senses good for given domain specificity. Because all nodes have already been adjusted to the domain in PABs, PABs can stay there in the trimmed model. On the other hand, a STM is such a sub-tree that an internal node is a root and the subordinates are only best-matched nodes. Because internal nodes have not been confirmed to have the senses good for a given domain, a STM can be moved in the trimmed model. Thus DODDLE II identifies PABs and STMs in the trimmed model automatically and then supports a user in constructing a conceptual hierarchy by moving STMs. Figure 3 illustrates the above-mentioned match result analysis.
In order to refine the trimmed model, DODDLE II can use trim result analysis as well as match result analysis. Taking some sibling nodes with the same parent node, there may be many differences about the number of trimmed nodes between them and the parent node. When such a big differences comes up on a sub-tree in the trimmed model, it may be better to change the structure of the sub-tree. DODDLE II asks the user if the sub-tree should be reconstructed or not. Based on empirical analysis, the sub-trees with two or more differences may be reconstructed. Figure 4 illustrates the abovementioned trimmed result analysis.
Finally DODDLE II completes taxonomic relationships of the input domain terms with hand-made additional modification from the user.
4 «‹£fi›fl§ › and is a 4-gram vector of a 4-gram . A set of vector ¢⁄£ƒ¥¤§ ¥ Here, is a vector representation of a word or phrase , ' £“¥¤§ ¥ is appearance places of a word or phrase in a text, 4. construction of word vectors A word vector is a sum of context vectors at all appearance places of a word or phrase within texts, and can be expressed with the follow formula.
is WordSpace. 3. construction of context vectors A context vector represents context of a word or phrase in a text. A sum of 4-gram vectors around appearance place of a word or phrase (called context area) is a context vector of a word or phrase in the place. ¡ 4-grams. Element in this matrix is the number of 4
trix is constructed in order to compare the context of two gram f which comes up just before 4-gram f¡ (called collocation area). The collocation matrix counts how many other 4-grams come up before the target 4-gram. Each column of this matrix is the 4-gram vector of the 4-gram f.
Non-taxonomic Relationship Learning almost comes from WordSpace[Marti and Schutze], which derives lexical cooccurrence information from a large text corpus and is a multi-dimension vector space (a set of vectors). The inner product between two word vectors works as the measure of their semantic relatedness. When two words inner product is beyond some upper bound, they are promising to have some non-taxonomic relationship between them. 1. extraction of high-frequency 4-grams Since letter-byletter co-occurrence information becomes too much and so often irrelevant, we take term-by-term co-occurrence information in four words (4-gram) as the primitive to make up co-occurrence matrix useful to represent context of a text. We take high frequency 4-grams in order to make up WordSpace. 4.1
WordSpace is constructed as shown in (Figure 5).
The best matched synset of each input terms in WordNet is already specified, and a sum of the word vector contained in these synsets is set to the vector representation of a concept corresponding to a input term. The concept label is the input term.
(1) close to 4.2
Vector representations of all concepts are obtained by constructing WordSpace. Similarity between concepts is obtained from inner products in all the combination of these vectors. Then, we define certain threshold for this similarity, and a concept pair with the similarity beyond it is extracted as a similar concept pair. A set of the similar concept pairs becomes concept specification templates. Both of concept pairs, which meaning is similar (with taxonomic relation), and which has something relevant each other (with non-taxonomic relation), are extracted as concept pairs with context similarity in a mass. However, by using taxonomic information from TRA module with co-occurrence information, DODDLE II distinguishes the concept pairs which hierarchically closes to each other from the other as TAXONOMY.
A user constructs a domain ontology by considering the relation with each concept pair in the concept specification templates, and deleting an unnecessary concept pair.
DODDLE II, domain ontology rapid development environment, which refer to MRD and domain-specific texts, is being implemented on Perl/Tk now. Figure 6 shows the ontology editor (left window) and the concept graph editor (right window). 5 5.1
In order to evaluate how DODDLE is doing in practical fields, case studies have been done in a particular law called Contracts for the International Sale of Goods (CISG). Two lawyers joined the case studies. In the first case study, input terms are 46 legal terms from CISG Part-II. In the second case study, they are 103 terms including general terms in an example case and legal terms from CISG articles related with the cases. One lawyer did the first case study and the other lawyer did the second.
Table 1 shows the case studies results. Figure 7 shows how much is included in final domain ontology the intermediate products at each DODDLE activity.
Generally speaking, in constructing legal ontologies, 70 % or more support comes from DODDLE. About half part of the final legal ontology results in the information extracted form WordNet. Because the two strategies just imply the part where concept drift may come up, the part generated by them has just about 30 % precision rate. Because the two strategies just take such syntactical feature as matched and trimmed results, the precision rate seems not to be so bad. In order to manage concept drift smartly, we will take into consideration the strategies with more semantic information that is not easy to come up in advance. 5.2
We have done the case study for learning non-taxonomic relationships in the field of CISG, taking 46 legal concepts from the above-mentioned case study. A user specifies nontaxonomic relationships, taking concept specification templates from DODDLE II
High-frequency 4-grams have been extracted from CISG (about 10,000 words). Duplications have been removed by doing standard form conversion. Thus we have got 526 kinds of 4-grams. In order to avoid sparseness of a collocation matrix to some exten, the extraction frequency of 4-grams must be adjusted according to the scale of text corpus. As CISG is comparatively small scale, it has been set to 8 times in this case study. The collocation matrix has been constructed by counting the number of each 526 kinds 4-gram just before a 4-gram for each kind. Since 526 kinds of 4-grams have been extracted, the collocation matrix have 526 dimensions. In order to construct a context vector, we have calculated the sum of 4-gram vectors around appearance place circumference of each of 46 concepts. One article of CISG consists of about 140 4-grams. The number of 4-gram vectors in context area has been set to 60 from an experience. For each of 46 concepts, the sum of context vectors in all the appearance places of the concept in CISG has been calculated and then the vector representation of the concepts has been obtained. The set of these vectors have been used as WordSpace to extract concept pairs with context similarity.
Having calculated the similarity from the inner product for the 1035 concept pairs that are all the combination of 46 concepts with the threshold of 0.9993, we have extracted 90 concept pairs and constructed concept specification templates based on them. Figure 8 illustrates the concept ”assent” specification template constructed by the user. In Figure 8, ”act” and ”proposal” have been identified as ancestor, descendant or a sibling of ”assent”, taking the concept hierarchical structure in the first case study. Thus the relationships have been distinguished as the label of TAXONOMY. As taxonomic relationships and non-taxonomic ones may come up together in the list of context similarity, it is useful to identify the taxonomic relationships by the concept hierarchical structure that has already been constructed. Given concept specification templates to a user, (s)he fills the kind of relationships in the templates, concept specifications have been done. Figure 9 illustrates the complete specification of the concept ”assent” from the concept specification template shown in Figure 8. assent non-TAXONOMY? : offeror
TAXONOMY : act non-TAXONOMY? : effect non-TAXONOMY? : offer non-TAXONOMY? : person non-TAXONOMY? : offeree non-TAXONOMY? : withdrawal non-TAXONOMY? : time
TAXONOMY : proposal The user with legal knowledge has evaluated how much the extraction of concept pairs has been done properly. The extracted concept pairs come up in Table 2.
Figure 10 shows the trade-off between precision and recall, changing the threshold of context similarity. In getting high hit rate, coverage is small. In getting high coverage, hit rate is low. Although proper threshold exists depending on taskdomains, it is hard to set it up in advance. We have not yet identified what relationship exists between two concepts. In order to do so, we need more information resources. using machine learning methods. They put together the nouns the prototypes of concept definition. Although small-scale
I have many thanks to Dr. Masaki Kurematsu for joining the case study and Mr. Maski Iwade for the implementations of DODDLE II.
“Wordnet”, also URL: case studies have been done in the field of law in order to see how DODDLE II is going in interaction with a user, we must do them in the large scale of case studies. We will consider how to take WEB content as the third information resources.