Building ontologies and debugging them is a timeconsuming task. Over the recent years, several approaches and tools for the automatic construction of ontologies from textual resources have been proposed. But, due to the limitations highlighted by experimentations in real-life applications, different researches focused on the identification and classification of the errors that affect the ontology quality. However, these classifications are incomplete and the error description is not yet standardized. In this paper we introduce a new framework providing standardized definitions which leads to a new error classification that removes ambiguities of the previous ones. Then, we focus on the quality of automatically built ontologies and we present experimental results of our analysis on an ontology automatically built by Text2Onto for the domain of composite materials manufacturing.
Since the pioneering works of Gruber [
The increasing popularity of ontologies and the scaling changes of
this last decade have motivated the development of ontology
learning techniques. Promising results have been obtained [
Whatever their use, it is essential to assess their quality
throughout their development. Several ontology quality criteria and
different evaluation methods have been proposed in the literature
[
In this paper, we focus on the most important errors that affect
the quality of semi-automatically built ontologies. To get closer the
operational concerns we propose a detailed typology of the different
types of problems that can be identified when evaluating an ontology.
Our typology is inspired from a generic standardized description of
the notion of quality in conceptual modeling [
The rest of this paper is organized as follows. Section 2 is a stateof-the art of the ontology errors. Section 3 describes a framework which provides a standardized description of the errors and draws correspondences between our new classification and the main errors previously identified in the literature. Section 4 presents our experimental results in the domain of composite materials manufacturing. More precisely, we analyze errors affecting an ontology produced by an automatic construction tool (here Text2Onto) from a set of technical textual resources. 2
In the literature, the notion of ”ontological error” is often used in a
broad sense covering a wide variety of problems which affect the
ontology quality. But, from several studies published this last decade,
we have identified four major denominations associated to
complementary definitions: (1) ”taxonomic errors” [
From the pioneering works of Gomez-Perrez [
Inconsistencies in the ontology may be logical or semantic. More precisely, three classes of inconsistencies in the taxonomic structure have been detailed: circularity errors (e.g. a concept that is a specialization or a generalization of itself), partitioning errors which produce logical inconsistencies (e.g. a concept defined as a specialization of two disjoint concepts), and semantic errors (e.g. a taxonomic relationship between two concepts that is not consistent with the semantics of the latter).
Incompleteness is met when concepts or relations of specialization are missing, or when some distributions of the instances of a concept between its sons are not stated as exhaustive and/or disjoint.
In the opposite way, redundancy errors are met when a taxonomic relationship can be directly deduced by logical inference from the other relationships of the ontology, or when concepts with the same father in the taxonomy do not share any common information (no instances, no children, no axioms, etc.) and can be only differentiated by their names. 2.2
Roughly speaking, design anomalies mainly focus on ontology understanding and maintainability. They are not necessarily errors but undesirable situations. Five classes of design anomalies have been described: (1) ”lazy concepts” (leaf concepts in the taxonomy not implied in any axiom and without any instances); (2) ”chains of inheritance” (long chains composed of intermediate concepts with a single child); (3) ”lonely disjoint” concepts (superfluous disjunction axiom between distant concepts in the taxonomy which may disrupt inference reasoning); (4) ”over-specific property range” (too specific property range which should be replaced by a coarser range which fits the considered domain better); (5) ”property clumps” (duplication of the same properties for a large set of concepts instead of the inheritance of these properties from a more general concept). 2.3
Ontology design patterns (ODP) are formal models of solutions
commonly used by domain experts to solve recurrent modeling problems.
Anti-patterns are ODP that are a priori known to produce
inconsistencies or unsuitable behaviors. [
Pitfalls are complementary to ODPs. Their broad definition covers
problems affecting the ontology quality for which ODPs are not
available. Poveda et al. [
It is easy to deduce from this classification that some pitfalls
should belong to different classes associated to different dimensions
(e.g. the fact that two inverse relations are not stated as inverse is
both a ”no inference” (NI) pitfall and an ”ontology understanding”
(OU) pitfall). Another attempt [
In order to highlight the links between the different classifications,
Poveda et al. tried to define a mapping between the classification in 7
classes deduced from the dimensions defined by Gangemi et al. [
The state of the art briefly presented in the previous section shows that the terminology used for describing the different problems impacting on the quality of ontologies is not yet standardized and that existing classifications do not cover the whole diversity of problems described in the literature.
In this section we present a framework providing standardized definitions for quality problems of ontologies and leading to a new classification of these problems. The framework comprises two distinct and orthogonal dimensions: errors vs. unsuitable situations (first dimension) and logical facet vs. social facet of problems (second dimension).
Unsuitable situations identify problems which do not prevent the usage of an ontology (within specific targeted domain and applications). On the contrary, errors identify problems preventing the usage of an ontology.
It is well known that one ontology has two distinct facets: an ontology can be processed by machines (according to its logical specification) and can be used by humans (including an implicit reference to a social sharing).
The remainder of the section is organized alongside the second dimension (i.e. logic vs. social facet) and within each facet, errors and unsuitable situations are defined. The framework is based on ”natural” analogies between respectively social and logical errors and social and logical unsuitable situations. 3.1 3.1.1
The logical ground problems can be formally defined by
considering notions defined by Guarino et al. [
Examples and formalizations hereinafter are provided by using a typical Description Logics notation (but easily transformable in first order or other logics).
The usual logical ground errors are listed below. 1. Logical inconsistency corresponding to ontologies containing logical contradictions for which a model does not exist (because the set of intended models is never empty, an ontology without models does not make sense anyway; formally, given an ontology O and the logical consequence relation according to the logical language L used for building O, there is no interpretation I of O such that I O). For example, if an ontology contains the following axioms B A (B is a A), A \ B > (A and B are disjoint), c B (c is instance of B), then c A and c A \ B, so there is a logical contradiction in the definition of this ontology; 2. Unadapted5 ontologies wrt to intended models6 i.e. an ontology for which something that is false in all (some of) the intended models of L is true in the ontology; formally, there exists a formula ' such that for each (for some) intended model(s) of L, ' is false and O '. For example, if we have in the ontology two concepts A and B that are declared as disjoint (O A \ B ?) and in each intended model there exists an instance c that is common between A and B (i.e. c A \ B), then the ontology is unadapted; 3. Incomplete ontologies wrt to intended models i.e. an ontology for which something that is true in all the intended models of L, is not necessarily true in all the models of O; formally, there exists a formula ' such that for each intended model of L, ' is true and O 2 '. As an example, if in all the intended models C [ B = A, and the ontology O defines B A and C A, it is not possible to prove that C [ B = A; 4. Incorrect (or unsound) reasoning wrt the logical consequence i.e. when some specific conclusions are derived by using suitable reasoning systems for targeted ontology applications even if these conclusions are not true in the intended models and must not be derived by any reasoning according to the targeted ontology applications (formally, when a specific formula ', false in the intended models O 2 ', can be derived O ` ' within any of those suitable reasoning systems); 5. Incomplete reasoning wrt the logical consequence i.e. when some specific conclusions cannot be derived by using suitable reasoning systems for targeted ontology applications even if these conclusions are true in intended models and must be derived by some 5 We use the term ”unadapted” instead of ”incorrect” ontologies because it remains unclear if intended models are defined for building the ontology or may also be defined independently. However, if intended models are defined for building the ontology, the term ”incorrect” may be more appropriate. 6 Intended models should have been defined fully and independently as in the case of models representing abstract structures or concepts such as numbers, processes, events, time and other ”upper concepts”, often defined according to their own properties. If intended models are not available, some specific entailments can be defined as facts that should necessarily be true in the targeted domain (or for targeted applications); specific counterexamples can also be defined instead of building entire intended models. reasoning according to the targeted ontology applications (formally, for some specific formula ', true in the intended models O ', cannot be derived O 0 ' within those suitable reasoning systems);
The most common logical ground unsuitable situations are listed below. These situations impact negatively on the ”non functional qualities” of ontologies such as reusability, maintainability, efficiency as defined in the ISO 9126 standard for software quality. 6. Logical equivalence of distinct artifacts (concepts / relationships / instances) i.e. whenever two distinct artifacts are proved to be logically equivalent; for example, A and B are two concepts in O and O A = B; 7. Symmetrically, logically indistinguishable artifacts i.e. whenever it is not possible to prove that two distinct artifacts are not equivalent from a logical point of view; in other words, if not possible to prove anyone of the following statements: (O A = B), (O A \ B ?) and (O c AandO c B); this case (7) can be partially covered in the case (3) above whenever intended models provide precise information on the equivalence or the difference between A and B; 8. OR artifacts i.e. an artifact A equivalent to a disjunction like C[S, A 6= C; S but for which, if applicable, it does not exist at least a common (non optional) role / property for C and S or because C and S have common instances; in the first case, a simple formalization can be expressed by saying that it does not exist a (non optional) role R such that O (C [ S) 9R:>; in the second case, an even simpler formalization is O c C and O c S, being c one constant not part of O; the first case targets potentially heterogeneous artifacts such as Car [ P erson, with probably no counterpart in the intended models, thus possibly leading to unadapted ontologies according to case (2) above; the second case targets potential ambiguities as, for instance, one role (property) R logically equivalent to a disjunction (R1 [ R2) being (R1 \ R2) satisfiable; 9. AND artifacts i.e. one artifact A equivalent to a conjunction like C \ S, A 6= C; S but for which, if applicable, it does not exist at least a common (non optional) role / property for C and S; this case is relevant to limit as much as possible some potentially heterogeneous artifacts such as Car \ P erson, possibly leading to artifact unsatisfiability; 10. While some case of unsatisfiability of ontology artifacts (concepts, roles, properties etc.) can be covered by (2) because intended models may not contain void concepts, unsatisfiability tout-court is not necessarily an error but a situation which is not suitable for ontology artifacts (i.e. given an ontology artifact A, O A ?); even if in ontologies it might be possible to define what must not be true (instead of what must be true), this practice is not encouraged; 11. High complexity of the reasoning task i.e. whenever something is expressed in a way that complicates the reasoning, while there exist more simple ways to express the same thing; 12. Ontology not minimal i.e. whenever the ontology contains unnecessary information:
Unnecessary because it can be derived or built7. An example of such unsuitable situation is the redundancy of taxonomic relations such as whenever A B, B C, and A C are all ontology axioms, the last axiom can be derived from the first two ones; 7 Built means that the artifact can be defined by using other artifacts. Unnecessary because it is not part of the intended models. For instance, a concept A being part of the ontology (language) but not defined by intended models. 3.1.2
Social ground problems are related to the perception (interpretation) and the targeted usage of ontologies by social actors (humans, applications based on social artifacts like WordNet, etc.). Perception (interpretation) and usage may not be formalized at all. In some sense, a further distinction between social facet and logical facet is as the distinction between respectively tacit and explicit knowledge.
There are four social ground errors: 1. Social contradiction i.e. the perception (interpretation) that the social actor gives to the ontology or to the ontology artifacts is in contradiction with the ontology axioms and their consequences; a natural analogy is with unadapted ontologies; 2. Perception of design errors i.e. the social actor perception accounts for some design errors such as modeling instances as concepts; a natural analogy is with unadapted ontologies; 3. Socially meaningless i.e. the social actor is unable to give any interpretation to the ontology or to ontology artifacts as in the case of artificial labels such as ”XYHG45”; a natural analogy is with unadapted ontologies; 4. Social incompleteness i.e. the social actor perception is that one or several artifacts (axioms and/or their consequences) are missing in the ontology; a natural analogy is with incomplete ontologies; The social ground unsuitable situations are mostly related to the difficulties that a social actor has to overcome for using the ontology especially due to limited understandability, learnability and compliance (as defined in ISO 9126). As for the logical ground unsuitable situations, it is difficult to dress an exhaustive list; the most common and important are listed below. 5. Lack of or poor textual explanations i.e. when there are few, no or poor annotations; prevents understanding by social actors; there are no natural analogies; 6. Potentially equivalent artifacts i.e. the social actors may identify as equivalent (similar) distinct artifacts as in the case of artifacts with synonymous or exactly the same labels assigned to distinct artifacts; a natural analogy is with logically equivalent artifacts; 7. Socially indistinguishable artifacts i.e. the social actors would not be able to distinguish two distinct artifacts as, for instance, in the case of artifacts with polysemic labels assigned to distinct artifacts; a natural analogy is with logically indistinguishable artifacts; 8. Artifacts with polysemic labels may be interpreted as union or intersection of their several rather distinct meanings associated to labels; a natural analogy is therefore with OR and AND artifacts. 9. Flatness of the ontology (or non modularity), i.e. ontology presented as a set of artifacts without any additional structure, especially if coupled with a important number of artifacts; a natural analogy is with high complexity of the reasoning task but also preventing effective learning and understanding by social actors; 10. Non-standard formalization of the ontology, using a very specific logics or theory, requires a specific effort by social actors for understanding and learning the ontology but also to use the ontology in standard contexts (reduced compliance); there are no natural analogies; 11. Lack of adapted and certified versions of the ontology in various languages requires specific efforts by social actors for understanding and learning the ontology but also to use the ontology in specific standard contexts (limited compliance); there are no natural analogies; 12. Socially useless artifacts included in the ontology; a natural analogy is with ontology not minimal. 3.2
The precise definitions of the proposed framework allow us to classify most of the ontology quality problems described in literature. Table 1 presents our classification of the different problems mentioned in Section 2. Some of the problems described in literature may correspond to more than one class of problems from our framework, as the definitions of these problems are often very large and sometimes ambiguous.
Table 1 reveals, at a first view, that the proposed framework provides additional problems that are not directly pointed out, to our knowledge, in the current literature about ontology quality and evaluation (but may be mentioned elsewhere). These problems are No adapted and certified ontology version, Indistinguishable artifacts,
correct reasoning. However, while covered, other problems are, in our opinion, too much narrowly defined in existing literature about ontology quality and evaluation. For instance, No standard formalization is specific to very simple situations while we refer to complete non standard theories.
A deeper analysis of Table 1 reveals that the ”logical anti-patterns”
presented in [
What is qualified as ”inconsistency” in [
None of the ”design anomalies” presented in [
Concerning ”pitfalls” [
Framework Logical inconsistency Unadapted ontologies Incomplete ontologies Incorrect reasoning Incomplete reasoning Logical equivalence of distinct artifacts Logically indistinguishable artifacts OR artifacts AND artifacts Unsatisfiability High complexity of the reasoning task Ontology not minimal Social contradiction Perception of design errors Socially meaningless Social incompleteness Lack/poor textual explanations Potentially equiv. artifacts Indistinguishable artifacts Polysemic labels Flatness of the ontology No standard formalization No adapted and certified ontology version Useless artifacts
State of the art problems â inconsistency error: ”partition errors - common instances in disjoint decomposition” â inconsistency errors: ”partition errors - common classes in disjoint decomposition”, ”semantic inconsistency” â logical anti-patterns: ”OnlynessIsLoneliness”, ”UniversalExistence”, ”AndIsOR”, ”EquivalenceIsDifference” â pitfalls: P5 (wrong inverse relationship, WI), P14 (misusing ”allValuesFrom”, MD), P15 (misusing ”not some”/”some not”, WI), P18 (specifying too much the domain / range, WI), P19 (swapping \ and [, WI) â incompleteness errors: ”incomplete concept classification”, ”disjoint / exhaustive knowledge omission” â pitfalls: P3 (”is a” instead of ”subclass-of”, MD), P9 (missing basic information, RC & RWM), P10 (missing disjointness, RWM), P11 (missing domain / range in prop., NI & OU), P12 (missing equiv. prop., NI & OU), P13 (missing inv. rel., NI & OU), P16 (misusing primitive and defined classes, NI) â pitfalls: P3 (using ”is a” instead of ”subclass-of”, MD), P24 - using recursive def., MD) â inconsistency error: ”circularity” â pitfall: P6 (cycles in the hierarchy, WI) â non logical anti-pattern: ”SynonymeOfEquivalence” â pitfall: P4 (unconnected ontology elements, RC) â design anomalies: ”lazy concepts” and ”chains of inheritance” â pitfall: P7 (merging concepts to form a class, MD & OU) â pitfall: P7 (merging concepts to form a class, MD & OU) inconsistency error: ”partition errors - common classes in disjoint decomposition” â logical anti-patterns: ”OnlynessIsLoneliness”, ”UniversalExistence”, ”AndIsOR”, ”EquivalenceIsDifference” â redundancy error: ”redundancy of taxonomic relations” â pitfalls: P3 (using ”is a” instead of ”subclass-of”, MD), P7 (merging concepts to form a class, MD & OU), P21 (miscellaneous class, MD) â non logical anti-pattern: ”SomeMeansAtLeastOne” â guidelines: ”Domain&CardinalityConstraints”, ”MinIsZero” â inconsistency error: ”semantic inconsistency” â logical anti-pattern: ”AndIsOR” â pitfalls: P1 (polysemic elements, MD), P5 (wrong inv. rel., WI), P14 (misusing ”allValuesFrom”, MD), P15 (misusing ”not some”/”some not”, WI), P19 (swapping \ and [, WI) â pitfalls: P17 (specializing too much the hierarchy, MD), P18 (specifying too much the domain / range, WI), P23 (using incorrectly ontology elements, MD) â non logical anti-pattern: ”SumOfSome” â design anomaly: ”lonely disjoints” â pitfalls: P12 (missing equiv. prop., NI & OU), P13 (missing inv. rel., NI & OU), P16 (misusing primitive and defined classes, NI) â pitfalls: P8 (missing annotation, OC & OU) â pitfalls: P2 (synonym as classes, MD & OU) â pitfalls: P1 (polysemic elements, MD & OU) â pitfalls: P20 (swapping label and comment, OU), P22 (using different naming criteria in the ontology, OC) â guidelines: ”GroupAxioms”, ”DisjointnessOfComplement” and ”Domain&CardinalityConstraints” â design anomaly: ”property clumps” â pitfall: P21 (using a miscellaneous class, MD & OU)
What problems are expected in automatically built ontologies. ment for building intended models, ad-hoc relations can be defined in the same way as standard relations. However, using standard reasoning it is expected (and even proved once fixing the logics) that reasoning algorithms are incomplete. However, adding artifacts may also solve some incompleteness and may also be useful for speeding up reasoning.
Only one of the seven classes of ”pitfalls” [
Most (16/20) of the pitfalls concerning the ”structural dimension”
of the ontology [
Although the proposed framework is general, we are especially concerned by ontologies automatically built from textual resources. We therefore aim at pointing the problems that are expected in automatically constructed ontologies (i.e. there is evidence of their presence or they will appear in future enrichments8 of the ontology). We are also interested by the opposite case, i.e. if there are unexpected problems in automatically constructed ontologies: it should be noted that unexpected problems are problems that even if the ontology may suffer of them, there is no evidence of their presence/absence for the ontology as it is (however, these problems may appear in future enrichments of the ontology). Our analysis is performed in two steps. In the first step (Section 4.1), we point out expected/unexpected problems due to inherent limitations of the tools for automatic ontology construction. In the second step (Section 4.2), we assess the results obtained in the first step by discussing our experience with the tool Text2Onto. 4.1
In a previous work [
Expected (Yes/No) and Why N (no axiom is defined ) contradictions are unexpected; but they remain possible in the case of future enrichments) Y (taxonomic relationships extraction algorithms are syntax based 6= from the intended models) Y (automatically extracted knowledge is limited to concepts and taxonomies 6= from the intended models) N (they might appear for complete formalization of concepts and relationships) Y (automatic tools consider that each term defines a different artifact ) the ontology may contain logically equivalent & logically indistinguishable artifacts) Y (polysemy of terms directly affects concepts / relationships: OR / AND concepts / relationships may appear) Y (polysemy of terms directly affects concepts / relationships: these latter may become unsatisfiable if their polysemic senses are combined) N (few or no axioms are defined ) reasoning remains very basic; but, it can be more complex if the ontology is further enriched) Y (automatic tools introduce redundancies in taxonomies) Y (ontologies are built from limited textual resources which may introduce contradiction in taxonomies) Y (the built ontology may contain concepts that are considered more close to instances by the social actor.) Y (several meaningless concepts with obscure labels are often introduced) Y (automatic tools consider that each term defines a different artifact ) distinct concepts can have synonymous labels ) these latter are perceived as potentially equivalent) Y (the ontology is incomplete ) it contains concepts that can be distinguished only by their labels; if such concepts have synonymous labels, they are indistinguishable) Y (automatic tools consider that each term defines a different artifact ) it is possible to have concepts with polysemic labels) Y (the ontology is poorly structured and has no design constraints - e.g. no disjunction axiom, lazy concepts) 4. Social incompleteness
Y (probably due to limited textual corpus) 5. Lack of or poor textual explanations
Y (usually automatic tools do not provide textual explanations) 10. No standard formalization
N (automatic tools usually can export their results in different formalization) 11. No adapted and certified ontology version 12. Useless artifacts
Y (automatically obtained results closely depend on the input texts language (often English) and certifying them is difficult) Y (automatic tools often generate useless artifacts from additional external resources)
During the last two years we were implied in a project called ISTA3 that proposed an ontology based solution for problems related to the integration of heterogeneous sources of information. The application domain was the management of the production of composite components for the aerospace industry. In this context, we tried to simplify the process of deploying the interoperability solution in new domains by using automatic solution for constructing the required ontologies.
The analysis presented in [
The textual resource used in the experiment presented in this paper is a technical glossary composed of 376 definitions of the most important terms of the domain of composite materials and how are they used for manufacturing pieces. The glossary contains 9500 words. For constructing the ontology we resort to the standard configuration for the different parameters of Text2Onto: all the proposed algorithms for concepts (and respectively for taxonomic relations) extractions have been used and their results have been combined with the default strategy.
The constructed ontology is an automatically built domain ontology that contains 965 concepts and 408 taxonomic relationships. Some of the central concepts of this ontology are: ”technique”, ”step”, ”compound”, ”fiber”, ”resin”, ”polymerization”, ”laminate”, ”substance”, ”form”. 4.2.2
Table 3 summarizes which types of problems have been identified in the automatically constructed ontology in our experience with Text2Onto. It also indicates, when possible, how many problems have been identified. Most of problems are relatively easy to identify and to quantify (e.g. the number of cycles in the taxonomical structure), but there are exceptions (e.g. the number of concepts or taxonomic relationships that are missing from the ontology). 4.2.3
No intended model or use case scenario was available when the expert analyzed the automatically constructed ontology. Consequently, it was able only to make a supposition concerning the logical completeness of the ontology and no logical error (unadapted ontology, incomplete or incorrect reasoning) was identified.
Few logical unsuitable situations are identified, but it is remarkable that they were identified automatically.
Unsurprisingly, most of the identified problems are social problems.
Yes: one taxonomical relationship can be deduced from two taxonomical relationships already present in the ontology (automatically identified by an ad-hoc algorithm) Yes: 15 taxonomic relationships are jugged semantically inconsistent by the expert Yes: 5 concepts that are interpreted as instances by the expert (units of measure and proper names) Yes: 21 concepts that have meaningless labels, for the expert Yes: no annotation associated to the ontology or to its artifacts Yes: 6 pairs of concepts have synonym labels, for the expert Yes: 69 concepts with polysemic labels, for the expert Yes: 389 lazy concepts lead to a poorly structured ontology Yes: 28 concepts are not necessary (3 are too generic, 25 are out of the domain) 6. Logical equivalence of distinct artifacts 7. Logically indistinguishable artifacts 8. OR artifacts 9. AND artifacts 10. Unsatisfiability 11. High complexity of the reasoning task 12. Ontology not minimal 1. Social contradiction 2. Perception of design errors 3. Social meaningless 4. Social incompleteness 5. Lack of or poor textual explanations 6. Potentially equivalent artifacts 7. Indistinguishable artifacts 8. Artifacts with polysemic labels 9. Flatness of the ontology 10. No standard formalization 11. No adapted and certified ontology version 12. Useless artifacts
No No No No No No Yes No No No
The analysis in Section 4.1 suggest that most of the problems that are expected in the automatically constructed ontologies are due to the fact that the automatic tool do not take into account the synonymy and the polysemy of terms when constructing concepts. However, even if Text2Onto, as configured for our test, do not group synonym terms when forming concepts, and allows polysemic terms to be labels for concepts, our test-case reveals that only two types of problems (socially indistinguishable artifacts and artifacts with polysemic labels) may be imputed to this limitation.
Most of the identified problems are related to the fact that the automatically constructed ontology seems to be incomplete. 5
In this paper, we have introduced a framework providing standardized definitions for different errors that have some impact on the quality of the ontologies. This framework aims at both unifying various error descriptions presented in the recent literature and completing them. It also leads to a new error classification that removes ambiguities of the previous ones. During ontology evaluation this framework may be used as a support for verifying in a systematic way if the ontology contains errors or unsuitable situations.
In the second part of the paper we focused on the quality of automatically built ontologies and we present experimental results of our analysis on an ontology automatically built by Text2Onto. The results show that a large part of the identified errors are linked to the ontology incompleteness. Moreover, it confirms that the identification of logical errors other than inconsistency requires intended models (or at least a set of positive and negative examples) and use case scenarii.
Due to the increasing complexity of the software, the identification
of the origin of each error in the ontology building process remains an
open question. And a further works consists in associating the
identified errors with the different tasks of an ontology construction (e.g.
the Methontology tasks [