The acquisition of knowledge from text is an incomplete and incremental process. When anchored to a particular knowledge model it provides potentially useful information to the legal expert in the form of new concepts and relations, in order to improve the domain coverage. This paper explores the feasibility of various legal text-based ontology enrichment techniques, and discusses the transformation of lexical knowledge to an ontological structure.
Ontology generation and population is a crucial part of knowledge base construction and maintenance that enables us to relate text to ontologies, providing on the one hand a customised ontology related to the data and domain with which we are concerned, and on the other hand a richer ontology, which can be used for a variety of semantic web-related tasks such as knowledge management, information retrieval and question answering.
Ontologies cover a particular knowledge domain in various levels of adequacy. Lacunae in domain coverage, different tasks or changes in the conceptualization require modifications of the ontology [1]. Ontology enrichment is a necessary ingredient of this ontology life cycle.
One source for enrichment of legal ontologies is the analysis of legal texts. It can generally be stated that law depends on language: regulatory knowledge must be communicated, and the written and oral transmission of social or legal rules passes through verbal expression. Therefore legal conceptual knowledge is closely related to language use within the legal domain. Legal discourse can never escape its own textuality [2], which implies that linguistic information plays an important role in its definition. In our work, we base ourselves on the postulation that there is, as in other terminological domains, a relatively high level of dependence between legal concepts and their linguistic realization in the various forms of legal language [3].
The acquisition of knowledge from resources such as texts is an incomplete and incremental process. Knowledge is quite often left implicit in text, or depends on previous analysis steps. This causes a sparseness problem for automatic acquisition. In our work we attempt to alleviate this problem firstly by bootstrapping and constraining the acquisition process on the basis of an existing legal ontology, which provides a solid conceptual framework. Secondly, perfect automatic knowledge acquisition does not exist. The acquisition results are considered informal suggestions that need expert evaluation and formalization into an enriched ontological structure as concepts and properties. These suggestions are necessarily partial and incremental. Their fragmented nature shows them as building blocks which, under expert supervision and according to an existing knowledge structure, enables the building and addition of knowledge in a bottom-up fashion.
This paper investigates the (semi-)automatic enrichment of a legal ontology by means of a selection of NLP techniques based on pattern matching and statistical analysis. It is exploratory in character and therefore its methodologies are only indicative of the potential of the applied techniques.
The main task we set ourselves is the investigation into the feasibility of ontology enrichment techniques. This ontology enrichment can take two forms. On the one hand, new relations between existing ontology elements may emerge from textual data. On the other, new candidate concepts with new relations with existing ontology elements may be suggested by an integrated linguistic and statistical text analysis.
Recently, many relation extraction approaches have been proposed focusing on the particular task of ontology development (learning, extension, population). These approaches aim to learn taxonomic or non-taxonomic relations between concepts, instead of lexical items. Therefore, the list of techniques applied in this paper is not exhaustive. It forms a subset of the full set of methodologies available. Most techniques described in this paper rely on robust and adaptable tools from the GATE architecture [4]. GATE is a framework for language engineering applications, which supports efficient and robust text processing. GATE uses NLP based techniques to assist the knowledge acquisition process for ontological domain modelling, applying automated linguistic analysis to create ontological knowledge from textual resources, or to assist ontology engineers and domain experts by means of semiautomatic techniques.
Our hypothesis is that the integration of corpus material, knowledge-based techniques and the use of rich linguistic processing strategies, can achieve effective results by accurately acquiring relevant relational knowledge [5]. A variety of techniques is helpful to the expert ontology engineer to extend the domain coverage of an existing ontology. 2
The DALOS domain ontology1 [10] aims to describe the domain of the consumer protection, which has been chosen as the pilot case in the recently finished DALOS project2, which resulted in the provision of support for the legal drafting process. It has been implemented as an extension of the Core Legal Ontology (CLO)3 developed on top of DOLCE foundational ontology [11] and on the “Descriptions and Situa
tions” (DnS) ontology [12] within the DOLCE+ library4. The extension covers the entities of the chosen domain and their legal specificities. In this network of ontologies the role of a core legal ontology is to describe concepts, which belong to the general theory of law, bridging the gap between domain-specific concepts and the abstract categories of formal upper level or foundational ontologies from DOLCE.
The domain ontology is populated by the conceptual entities which characterize the consumer protection domain. Such domain-specific concepts are classified according to more general notions, imported from CLO, as Legal role and Legal situation. Examples of consumer law concepts are CommercialTransaction, Consumer, Supplier, Good and Price. The first version of the DALOS Ontological layer contains 121 named classes. 3
The DALOS ontology is the result of a manual effort within the DALOS project. Ontological modelling of legal domains is a constant effort. Domain descriptions need to be refined. Legislation evolves in the sense that new directives are issued, and old ones are deprecated. Therefore its coverage of the domain of consumer protection in terms of ontological vocabulary is never complete, and should be constantly adapted on the basis of expert advice and data-driven suggestions. Its incorporation of top level ontologies such as DOLCE make it descriptively adequate and robust for the higher levels of ontological legal description, but in terms of fine-grained domainspecific vocabulary it continuously remains in need of refinement and extension.
Our aim is to provide data-driven suggestions for ontology extension in the form of lexical material from the English legal texts in the DALOS corpus, which consists of directives and judgements (270,000 words in 55 directives and judgements). The results carry no more authority than suggestions for expert evaluation. For our analyses described below, the evaluator is a computational linguist, not a legal expert.
The main task these analyses perform is the general knowledge based identification of text-derived information that is of possible interest for legal ontology enrichment. Legal relevance will be an additional evaluation phase in which the data, deemed relevant from a general perspective, are assessed by an expert, and, if deemed relevant for the legal knowledge expressed by the DALOS ontology, integrated into an extended knowledge structure. 4
The idea of acquiring semantic information from texts dates back to the early 1960s with Harris' distributional hypothesis [7] and Hirschman and Sager's work in the 1970s [8], which focused on determining sets of sublanguage-specific word classes using syntactic patterns from domain-specific corpora. Many techniques have since been proposed for the task of extracting knowledge from texts. Overall, the majority
4.1
of approaches can be divided into pattern-based (pattern matching in a corpus) and statistically-based extraction [21]. Quite often, the two techniques are mixed (e.g. [24], [25], [26]). A description of several other approaches for conceptual relation extraction aiming at ontology learning can be found in [9].
The GATE platform5 forms the methodological basis for our work [4]. A number of tools have been developed and used for the task of legal ontology enrichment. They all rely on the initial stage of linguistic pre-processing the corpus under examination, in order to obtain valuable linguistic information that will be used in later processing. 4.2
First, tokenization and sentence splitting divide up the text into manageable units. Then part of speech tagging and lemmatization allow the inclusion of morpho-syntax into the analysis. 4.3
The extraction tool TermRaider produces term candidates from a corpus by first filtering out possible terms by means of a multi word unit grammar that defines the sequences of part of speech tags constituting noun phrases. The computation of term frequency/inverted document frequency (TF/IDF) [13] [20], a technique widely used in information retrieval and text mining taking into account term frequency and the number of documents in the collection, yields a score that indicates the salience of term candidates for each document in the corpus. All term candidates with a TF/IDF score higher than an empirically determined threshold are then selected. 4.4
Lexico-syntactic patterns are textual patterns that, with morphosyntactic normalization such as lemmatization, are highly indicative of semantic relations between textual elements. Ontology population based on this pattern approach has proven to be reasonably successful for a variety of tasks [6].
The following pattern matching strategies have been applied:
This technique looks for a match between a pair of elements, of which one is embedded into the other as the head of a syntactic construction. The ontological interpretation of this relation is the insertion of a hyponymic relation between these elements. Examples from the Dalos ontology are: 5 www.gate.ac.uk
Activity SuperClassOff CommercialActivity Ten head matching relations were found in the ontology. All ten are covered in the ontology by means of superclass relations, except for one: Agent isSuperClassOff PhysicalAgent. PhysicalAgent is an object, and Agent is a top concept. PhysicalAgent is a hypernym of NaturalPerson, and the definition of Agent is: “A natural or legal person which plays the role of legal subject“. We can therefore conclude on the basis of this definition that this additional a subsumption relation holds.
Matching the term candidates identified by TermRaider with existing classes resulted in 378 matching pairs. Manual evaluation of this set showed that 115 (around 30%) of them should be considered by experts for possible inclusion into the DALOS ontology. As an illustration, the following candidate subclasses of Contract were extracted, which show the detail of terminological specification in this domain: timeshare contract; purchase contract; credit contract; package travel contract; consumer contract; building contract.
The second acquisition technique is based on Hearst patterns [14], which are a set of lexico-syntactic patterns that indicate hyponymic relations, and have been widely used by other researchers. Typically, they achieve a very high level of precision, but quite low recall [21]: in other words, they are very accurate but only cover a small subset of the possible patterns for finding hyponyms and hypernyms. The patterns can be described by the following rules, where NP stands for a Noun Phrase and the regular expression symbols have their usual meanings6: { NP such as (NP,)* (or|and) NP Example: “advertising and marketing practises, such as product placement, brand differentiation or the offering of incentives…” { NP (,NP)* (,)? (or|and) (other|another) NP Example: “…whereby a creditor grants or promises to grant to a consumer a credit in the form of a deferred payment, a loan or other similar financial accommodation.” No matching patterns between Dalos ontology elements were found. Table 1 below lists the results for obtained patterns between term candidates selected by TermRaider and Dalos ontology elements. The success rate is lower than expected (27% on average), given the reported high precision of Hearst patterns.
Hearst Pattern Such as Including And other Or other Especially Number found 31 0 0 2 1
Whereas bot a) and b) produce paradigmatic (isa) relations between terms, pointwise mutual information7 (MI) is a well-known technique that measures the mutual dependence of the two variables as an expression of a syntagmatic relation. It is commonly used as a significance function for the computation of collocations in corpus linguistics [15]. In our case, it measures the statistically-based strength of relatedness through collocation within the same document.
Overall, forty MI relations were found between existing concepts from the Dalos ontology after matching DALOS ontology labels onto textual elements. Nine (22.5%) of the forty are not connected by any relation or concatenation of relations in the ontology. For example, the following pairs with their MI value:
Seller ConsumerGoods ImmovableProperty ImmovableProperty FinancialService
ConsumerGoods Producer Contract FinancialService Product
A number of concepts (Consumer, Supplier, Trader, Producer, Organizer and Seller) are all subconcepts of LegalRole in the DALOS ontology. As co-hyponyms they are 7 See http://www.collocations.de/ and http://en.wikipedia.org/wiki/Mutual_information
Thirty one (77.5%) are related within the ontology, expressed by property concatenations in varying degrees of complexity.
Six MI pairs have a direct connection between its members, as illustrated below:
Advertising isAbout Product CreditAgreement hasParticipant Consumer not directly related, but indirectly through their hypernym. The 11 MI pairs in which they are collocations seem to express ontological relations that are applicable to this whole set of co-hyponyms, in varying property configurations, such as Contract and CreditAgreement, of which Contract is the strongest indicator.
Contract Consumer Consumer CreditAgreement Supplier Seller DistanceContract Trader Contract Contract Supplier
Organizer Supplier Seller Supplier Contract Contract Consumer Consumer Consumer FinancialService Producer
The remaining fourteen of the MI concept pairs have complex indirect links between them, which consist of a concatenation of object properties. For example: Producer Product 4.21 Product isObjectOf Advertising Isactedin CommercialTransaction hasParticipant Agent hasRole Producer Consumer CommercialCommunication 3.37 CommercialCommunication isActedIn CommercialTransaction hasParticipant Consumer Consumer GeographicalAddress 2.45 GeographicalAddress isQualityOf NaturalPerson hasConsumerRole Consumer Product Consumer 0.21 Product isObjectOf Advertising Isactedin CommercialTransaction hasParticipant Agent hasRole Consumer
These results indicate the potential for statistical techniques - in this case the computation of mutual information values for pairs of ontology members- for the identification of fine-grained relations between concepts. 77.5% of the extracted MI relations are already attested in the ontology. The 22.5% of the MI pairs without ontological confirmation make ontological sense to the inexpert eye in that they express finegrained relations that should be expertly evaluated for inclusion into the ontology, and linked to existing ontology elements by means of existing or new object properties.
The value of the MI score does not seem to matter much in terms of validity of a relation between the ontology elements, nor does it seem indicative of the length of the path between the ontology elements. The actual detection of a relation by means of MI computation seems to be crucial in this case, and it is up to experts to determine the granularity of the property vocabulary in the ontology, and decide whether this relation needs to be made explicit by means of one object property, or a concatenation of object properties.
Verbal patterns typically reflect lexicalized semantic relations between its arguments. Patterns defined in GATE can consist of any type of annotation that has been added in GATE, e.g. part of speech, string value, lemma etc. The corpus indexing and querying tool in GATE, called ANNIC8 (ANNotations In Context) [16], allows the evaluator to enter search patterns over text annotations, and detect semantic relations between ontology elements at the fine-grained text level.
As proof of concept, the following simple pattern was defined, which identifies pairs of elements from the Dalos ontology that are mentioned in the texts as verb arguments. The surface representation restricts the verb context to a two-token window on either side. {DalosConcept}({Token})*2{Token.category=="VERB"}({Token})*2 cept} {DalosConA graphical user interface allows the user to query a corpus and inspect the results from the query. The screenshot in Figure 1 below illustrates how the results are displayed in the GATE interface. Annotations over spans of text are displayed as rows with coloured blocks indicating part of speech, string and DalosConcept. Contexts to the left and right of the text matching the search pattern are displayed at the bottom.
Using this query, 56 patterns were extracted, of which 37 (66%) were evaluated as deserving expert attention. For example:
NaturalPerson Seller/Supplier Consumer Consumer Consumer Consumer CompetentAuthority conclude buy dissolve enter into purchase rely on acquire assess
Product Contract CreditAgreement Product Guarantee Services Product with Seller or Supplier 8 http://videolectures.net/gate06_aswany_ac/
Surface patterns and text spans are potential lexical realizations of underlying ontological relations and concepts. Ontologies themselves are conceptual constructs without linguistics. From a formal ontological point of view, concepts are abstract notions whose labels (often constituted by textual elements) are arbitrary. The lexical senses of the lexicalizations that function as labels for these concepts, are only considered to be evocative or indicative of the ontological meaning of the concepts. There is an implicit mapping assumption between lexical and conceptual knowledge, which underlies "ontology lexicalization", namely that (intensional) senses from a lexical model are mapped to (extensional) interpretations on ontology elements (individuals, classes, restrictions, properties) [17].
The reification of lexical material into ontological elements can happen in various ways. Some authors state that there is a direct relation between lexical form and surface syntactic pattern and ontological content [18]. Others advocate a formalization process that transforms surface patterns into ontology concepts and object properties in a number of stages, maintaining the philosophical distinction between lexical meaning and conceptualization, and allowing predication over these various levels of semantic representation [19].
The first stage is a transformation of linguistic elements (abstracted away from surface forms by means of lemmatization and other linguistic normalization processes such as morphological decomposition) into a semantic metamodel, which expresses the semantics of the domain. The next step, the transformation of this semantic domain knowledge into an ontological representation language construct such as OWL9, 9 http://www.w3.org/2004/OWL decides on the ontological status of the semantic knowledge, e.g. whether it should be encoded as class, an attribute of an object property. 6
When applying a variety of NLP techniques for ontology extension, each technique provides its specific spectrum of potential ontological enrichment based on the nature of the linguistic and statistical algorithms involved. Overall, the four acquisition techniques described in this paper (head matching, Hearst patterns, mutual information and simple verb complementation patterns) form a representative combination of acquisition techniques for both paradigmatic and syntagmatic lexical semantic relations. They perform reasonably well for establishing relations between ontology elements (81.2% average success rate excluding Hearst patterns, for which no hits were found). Since Hearst patterns are very sparse at best, future work on text-based ontological relation acquisition will look at the extension of the Hearst pattern set with more textual patterns reflecting the paradigmatic isa-relation (taken from e.g. [27]).
Head matching and Hearst patterns between term candidates and ontology elements have an average success rate of 28.5%, which is lower than expected. Overall, we can conclude that the techniques work well for identifying relations between ontology elements.
The reification of these surface syntactic and collocational relations may take several forms, depending on the strategy chosen. For some of the extracted relations based on verbal and deverbal lexicalizations, the proposed corresponding ontological relations are not always disjoint. For instance, in a number of cases, it is possible to group certain relations together under synonymy. As an example, the textual fragments “supply of services” and “provision of services” contain deverbal nouns, which, when translated into verbal counterparts, yield the following object properties:
AGENT provide SERVICE
Since “supply” and “provide” are synonyms in WordNet [22], the object can be
renamed into a common label, which covers both verbal lexicalizations. Further
mapping with lexical resources such as VerbNet [23] will further classify the relations into
more general classes, and provide semantic role arguments (e.g. agent, instrument
etc.). Together with further analysis of the lexicalizations that instantiate these
patterns, this will lead to an incremental creation of semantic frames, which then can be
transformed into their ontological counterparts with ontologically proper constraints
on the domain and range of the reified properties.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
Guarino, N., Social roles and their descriptions, In: Welty, C. (ed.), Proceedings of
the Ninth International Conference on the Principles of Knowledge Representation
and Reason