Ontology Learning from text aims at generating domain ontologies from textual resources by applying natural language processing and machine learning techniques. It is inherent in the ontology learning process that the acquired ontologies represent uncertain and possibly contradicting knowledge. From a logical perspective, the learned ontologies are potentially inconsistent knowledge bases that thus do not allow meaningful reasoning directly. In this paper we present an approach to generate consistent OWL ontologies from learned ontology models by taking the uncertainty of the knowledge into account. We further present evaluation results from experiments with ontologies learned from a Digital Library.
Ontology Learning from text aims at generating domain ontologies from a given collection of textual resources by applying natural language processing and machine learning techniques. Due to an increasing demand for efficient support in knowledge acquisition, a number of tools for automatic or semi-automatic ontology learning have been developed during the last years. Common to all of them is the need for handling the uncertainty which is inherent in any kind of knowledge acquisition process. Moreover, ontology-based applications which rely on learned ontologies have to face the challenge of reasoning with large amounts of imperfect information resulting from automatic ontology generation systems.
Causes for the imperfection of information can be found thrice. According to [
In ontology learning, (subjective) uncertainty is the most prominent form of imperfection. This is due to the fact that the results of the different algorithms have to be considered as unreliable or irrelevant due to imprecision and errors introduced during the ontology generation process. There exist different approaches for the representation of uncertainty: Uncertainty can for example be represented as part of the learned ontologies, e.g. using probabilistic extensions to the target knowledge representation formalism, or at a meta-level as application-specific information associated with the learned structures.
In Text2Onto [
An obvious alternative approach to dealing with potential inconsistencies is to
prohibit primitives that introduce inconsistencies in the first place (e.g. negation,
disjointness). However, as shown in [
As a main contribution of this work we present a transformation that results in an ontology that is (1) consistent and (2) “most likely correct”, relying on the certainty information of the LOM model. The transformation is based on the notion of an evaluation function that measures the quality of ontologies with respect to given criteria, i.e. in our case consistency and certainty.
Application Scenario Intelligent search over document corpora in Digital Libraries is one application scenario that shows the immediate benefit of the ability to reason over ontologies automatically learned from text. While search in Digital Libraries nowadays is restricted to structured queries against the bibliographic metadata (author, title, etc.) and to unstructured keyword-based queries over the full text documents, complex queries that involve reasoning over the knowledge present in the documents are not possible. Ontology learning enables obtaining the required formal representations of the knowledge available in the corpus to be able to support such advanced types of search. This application scenario is the subject of a case study within the Digital Library of BT (British Telecom) as part of the SEKT1 project. One of the key elements
of the case study is to automatically learn ontologies to enhance search and finally be able support queries of the kind “Find knowledge management applications that support Peer-to-Peer knowledge sharing.” To validate the work the presented in this paper, we performed experiments with data from the BT Digital Library.
Overview of the paper The rest of the paper is organized as follows. In Section 2 we recapitulate the foundations of the OWL ontology language, query answering with OWL ontologies and the role of logical inconsistencies. In Section 3 we introduce the Learned Ontology Model (LOM). In Section 4 we discuss the transformation of LOM models to OWL ontologies. We discuss experimental results in Section 5 and present related work in Section 6 before we conclude in Section 7. 2
In this section we provide on overview of the OWL ontology language (specifically OWL-DL), typical reasoning tasks and show why standard reasoning with inconsistent ontologies does not yield meaningful results.
OWL-DL is a syntactic variant of the SHOIN (D) description logic [
Definition 1 (Ontology). We use a datatype theory D, a set of concept names NC , sets of abstract and concrete individuals NIa and NIc , respectively, and sets of abstract and concrete role names NRa and NRc , respectively.
The set of SHOIN (D) concepts is defined by the following syntactic rules, where A is an atomic concept, R is an abstract role, S is an abstract simple role, T(i) are concrete roles, d is a concrete domain predicate, ai and ci are abstract and concrete individuals, respectively, and n is a non-negative integer:
C ! A j :C j C1 u C2 j C1 t C2 j 9R:C j 8R:C j ¸ n S j · n S j fa1; : : : ; ang j j ¸ n T j · n T j 9T1; : : : ; Tn:D j 8T1; : : : ; Tn:D D ! d j fc1; : : : ; cng A SHOIN (D) ontology O is a finite set of axioms of the form concept inclusion axioms C v D, for C and D concepts, transitivity axioms Trans(R), role inclusion axioms R v S and T v U , concept assertions C(a), role assertions R(a; b), individual (in)equalities a ¼ b, and a 6¼ b, respectively.
The semantics of the SHOIN (D) description logic is defined via a model-theoretic semantics, which explicates the relationship between the language syntax and the model of a domain: An interpretation I = (4 ; ¢I ) consists of a domain set 4I , disjoint from
I the datatype domain 4ID, and an interpretation function ¢I , which maps from individuals, concepts and roles to elements of the domain, subsets of the domain and binary relations on the domain, respectively2. An interpretation I satisfies an ontology O, if it
satisfies each axiom in O. Axioms thus result in semantic conditions on the interpretations. Consequently, contradicting axioms will allow no possible interpretations. This leads us to the definition of a consistent ontology: Definition 2 (Consistent Ontology). An ontology O is consistent iff O is satisfiable, i.e. if O has a model.
To be able to define queries against ontologies, we rely on the notion of entailment: We use O j= ® to denote that the ontology O entails the axiom ® (alternatively, we say that ® is a consequence of the ontology O), iff ® holds in any model in which O holds. Definition 3 (Query and Query Answer). A query with respect to an entailment relation j= is a pair of an ontology O and an axiom ®, written ’O j= ®?’. An answer to a query ’O j= ®?’ is a value in the set ftrue; f alseg as O j= ® and O 6j= ® respectively.
Standard entailment as defined above is explosive, i.e. any axiom is a consequence
of an inconsistent ontology. Namely, if an ontology O is not consistent, then for any
axiom ®, O j= ®. In other words, query answers for inconsistent ontologies are
completely meaniningless, as for any query the query answer will be true. For a detailed
discussion on inconsistencies in OWL ontologies, we refer the reader to [
We believe, that linguistic evidence with respect to an ontology can be appropriately
measured by ontology learning techniques which try to capture the ontological
commitment in human language. Since ontology learning algorithms such as implemented
in TextToOnto [
A Learned Ontology Model (LOM) as used by Text2Onto is a collection of instantiated modeling primitives which are independent of a concrete ontology representation language. These primitives are defined in a declarative fashion which allows for translating the LOM into any knowledge representation language as long as the expressivity of the primitives does not exceed the expressivity of the target language. In Text2Onto we follow a translation-based approach to knowledge engineering. So called ontology writers are then responsible for translating instantiated modeling primitives into a specific target knowledge representation language. While a translation to various ontology languages is possible, in the scope of this paper, we focus on the translation to OWL ontologies. The modeling primitives we use in Text2Onto and their correspondences in the OWL ontology model are described by Table 1.
To capture contextual information about ontology elements, such as provenance and certainty in the learning process, we introduce the notion of rating annotations. to indicate how confident the system is about the correctness of an ontology element. The confidences are calculated based on different kinds of evidences provided by the ontology learning algorithms that indicate the correctness and the relevance of ontology elements for the domain in question. They can be considered as a corpus-based support for ontology elements.
Algorithms We now describe for each modeling primitive the algorithms used to learn corresponding instances thereof. In particular, we explain the way the confidence and relevance ratings for an instantiated modeling primitive are calculated.
Concepts and Instances Different term weighting measures are used to compute
the relevance of a certain concept or instance with respect to the corpus: Relative Term
Frequency (RTF), TFIDF, Entropy and the C-value/NC-value method in [
Subconcept-of Relations In order to learn subconcept-of relations, we have
implemented a variety of different algorithms exploiting the hypernym structure of WordNet
[
Instance-of Relations In order to assign instances or named entities appearing in
the corpus to a concept in the ontology Text2Onto relies on a similarity-based approach
extracting context vectors for instances and concepts from the text collection and
assigning instances to the concept corresponding to the vector with the highest
similarity with respect to their own vector [
General Relations To learn general relations, Text2Onto employs a shallow
parsing strategy to extract subcategorization frames (e.g. hit(subj,obj,pp(with)),
transitive + PP-complement) enriched with information about the frequency of the terms
appearing as arguments [
Equivalence and Equality Following the assumption that terms are similar to the extent to which they share similar syntactic contexts, we implemented algorithms calculating the similarity between terms on the basis of contextual features extracted from the corpus, whereby the context of a terms varies from simple word windows to linguistic features extracted with a shallow parser. This corpus-based similarity is then taken as the confidence for the equivalence of the corresponding concepts or instances.
Disjointness For the extraction of disjointness axioms we implemented a simple heuristic based on lexico-syntactic patterns. In particular, given an enumeration of noun phrases N P1; N P2; :::(andjor)N Pn we conclude that the concepts C1; C2; :::Ck denoted by these noun phrases are pairwise disjoint, where the confidence for the disjointness of two concepts is obtained from the number of evidences found for their disjointness in relation to the total number of evidences for the disjointness of these concepts with other concepts. 4
In this section we discuss the transformation of learned ontologies as described in the previous section to OWL ontologies (c.f. Section 2). As mentioned before, a naive translation that simply disregards the certainty information (rating annotations) would result in a potentially highly inconsistent knowledge base that would not allow meaningful reasoning. The goal of the transformation therefore is to obtain an ontology that is (1) consistent (to allow meaningful reasoning), and (2) captures the most certain information while disregarding the potentially erronous information. In general, there may be many different consistent ontologies obtained from a LOM. The difficulty is to select the “best” ontology, i.e. the one that will result in most meaningful reasoning. Evaluation Function In order to able to define what a “good” ontology for a particular context is, we need to be able to measure the quality of the ontology with respect to given set of criteria. We therefore define the notion of an ontology evaluation function. ecertainty(O) =
0 (max ³ P®2O krOcoknf (®)¡t ; 0´ if O is consistent Definition 5. Let O be the set of possible ontologies, then an ontology evaluation function e is a function e : O ! [0; 1].
Effectively, the evaluation function provides a total order over the space of possible ontologies and thus allows to compare given ontologies. Here it is important to note that the evaluation function can take the rating annotations into account and thus provides an evaluation measure for a given context. Using the evaluation function, we can define the problem of translating a given learned ontology LOM to a “discrete” and consistent OWL ontology as: maxOµLOM e(O).
In other words, we try to find the best ontology O based on the knowledge in LOM that maximizes the evaluation function.
For our particular goal to obtain a consistent ontology capturing the most certain information, we can define an evaluation function as follows:
Let us discuss the intuition behind this function. The basic idea is to maximize the certainty of the ontology based on the confidence of its individual axioms, as given by rconf (®). The threshold t is introduced to “filter out” axioms with a confidence below a minimal value: Adding an axiom with a confidence below t will thus decrease the value of ontology. An inconsistent ontology is defined to have “no value”.
In general, it will be hard to determine the optimal ontology that maximizes the evaluation function, as one theoretically would need to search entire space of possible consistent ontologies. However, in most cases it is not necessary to prove the optimality of an obtained solution, especially when considering that the rating annotations themselves are already somewhat imprecise. Instead it is possible to exploit heuristics to obtain a “fairly” optimal ontology.
We now outline an algorithm that exploits the behavior of the evaluation function
and local characteristics of inconsistencies to maximize the value. It is based on the
ideas of consistent ontology evolution as presented in [
We treat the transformation of a LOM ontology to a consistent OWL ontology in
a similar way as shown in Algorithm 1: Starting with an empty ontology O, we
incrementally add all axioms from the learned ontology LOM whose confidence is equal to
or greater than the threshold t. If adding the axioms leads to an inconsistent ontology,
we localize the inconsistency by identifying a minimal inconsistent subontology. (For
the details of this procedure, we refer the reader to [
For the transformation of the LOM ontology to a discrete OWL ontology, we applied the evaluation function and algorithms presented in the previous section. Here we performed an analysis of the influence of the threshold of uncertainty on the transformation. The results in Table 2 clearly show the connection between the level of uncertainty and inconsistency introduced:
Threshold t # of Inconsistencies # of Axioms in Result 0.1 0.2 0.4 0.8
A low threshold t results in more uncertain information being allowed in the target ontology. As a result, the chances for inconsistencies increase. How to choose the “right” threshold t for the transformation process will very much depend on the application scenario, as it essentially means finding a trade-off between the amount of information learned and the confidence in the correctness of the learned information.
In the following we will discuss typical types of inconsistencies and present examples of such inconsistencies that were detected and resolved. The first type of inconsistency involves unsatisfiable concepts (often called incoherent concepts) in the T -Box of the ontology. This can for example happen if two concepts are identified to be disjoint, but at the same time these concepts are in a subconcept-relation (either explicitly asserted or inferred). Interestingly, this type of inconsistency often occurred for concepts for which even for a domain expert the correct relationship is hard to identify, as the following example shows: Example 1. The relationship between the concepts Data, Information, and Knowledge is a very subtle (often philosophical) one, for which one will encounter different definitions depending on the context. The (inconsistent) definitions learned from our data set stated that Data is a subconcept of both Inf ormation and Knowledge, while Inf ormation and Knowledge are disjoint concepts:
Data v Inf ormation Data v Knowledge Inf ormation v :Knowledge
The inconsistency was resolved by removing the disjointness axiom, as its confidence value was lowest.
The second type of inconsistencies involves A-Box assertions. Here, typically instances
were asserted to be instances of two concepts that were identified to be disjoint. We
again present an example:
Example 2. Here KaV iDo was identified to be both an instance of Application and a
T ool (based on the abstract of [
Application(kavido) T ool(kavido) T ool v :Application
This inconsistency was again resolved by removing the disjointness axiom.
Other types of inconsistencies involving, for example, domain and range restrictions were not considered in our current experiments, thus being left for future work. Nevertheless, this evaluation showed that inconsistency is an important issue in ontology learning. 6
Since building an ontology for a huge amount of data is a difficult and time consuming
task a number of tools such as TextToOnto [
The approach to dealing with inconsistencies presented in this work is based on the
idea of obtaining a consistent ontology from a LOM to be then able to derive consistent
query answers. A very related approach is that of reasoning with inconsistent
ontologies. A typical technique is the selection of a consistent subontology for a given query,
which yields a consistent query answer (c.f. [
Ontology learning is a promising technique for automated knowledge acquisition from text corpora. However, as we have shown, uncertainty and inconsistencies are issues that need to be dealt with in order to allow meaningful reasoning over the learned ontologies. In this paper we have presented how uncertainty can be represented in the Learned Ontology Model (LOM) and how such learned ontologies can be transformed to consistent OWL ontologies using the notion of an ontology evaluation function. Our experiments with ontologies learned from documents of a Digital Library show the feasibility and usefulness of the approach. An extensive evaluation will be performed as part of a case study within the SEKT project.
It is important to mention that confidence as generated by ontology learning
algorithms represent a data-driven approach to the evaluation of ontologies. There are many
other notions of ontology quality and consistency which could be used for the definition
of an ontology evolution function. In particular, we will in the future integrate an
automatic approach towards the formal evaluation of ontologies by means of the OntoClean
methodology as presented in [
Acknowledgements Research reported in this paper has been financed by the EU in the IST project SEKT (IST-2003-506826) (http://www.sekt-project.com/).