A novel Information Extraction system able to generate complex instances from free texts available on the Web is presented in this paper. The approach is based on non monotonical processing over ontologies, and makes use of entity recognizers and disambiguators in order to adequately extract and combine instances and relations between them. Experiments conducted over the archaeological research domain provide satisfactory results and suggest that the tool is suitable for its application on Semantic Web resources.
The Semantic Web is a form of web conceived for allowing human users and software tools to process and share the same sources of information. It builds on a set of standards which ensure syntactic consistency and semantic value. Large communities are participating in its development, producing as a result huge domain ontologies with very rich lexicons. Consequently, the problem of identifying possible instances of these ontologies, usually called semantic annotation, ontology population or instance extraction, has become crucial.
The formalization of a framework for the consistent generation of complex instances for the Semantic Web is the main contribution of this paper. A complex instance is an ontological instance involving several levels of aggregation between the entities mentioned in the document. Complex instances can also consist of summaries of sets of instances (i.e. generalized instances) and can include negative properties in their definition. To the best of our knowledge, no published work deals with this kind of instances, although many methods are available for extracting plain facts from text chunks.
Our system is able to collect complex instances and their relationships across the whole document, using the structure of the document and the relations between concepts expressed through the ontology. A non monotonic processing ∗This work has been partially funded by the “Juan de la Cierva” program of the Ministry of Education and Science of Spain. through an initial set of instances allows one to update the initial knowledge base according to the semantic descriptions of the ontology. Contradictory or inconsistent data can be instead removed from the knowledge base.
Section 2 of this paper reviews the current approaches for automatic ontology population. Section 3 provides a global description of our approach, which is formalized in Sections 4 and 5. Section 6 describes the practical process of instance extraction, which experimental results are presented in Section 7. 2
Semantic annotation methods can be classified into two fundamental types [
Most annotation methods require a complete syntactic analysis, and in some cases a semantic analysis too, including co-reference and anaphora resolutions. Current syntactic parsers are error prone and their performance is often not satisfactory even for medium-size document collections. These methods are thus generally not suitable for the large scale analysis required by the Semantic Web (with the exception of probabilistic methods, which avoid syntactic analysis). Methods based on manual definition of rules require moreover a constant updating and a customized adjustment to each scenario.
The use of training corpora, such as in machine learning methods, is another
source of complexity. In fact, corpora creation is a very resource consuming task,
and learned models depend heavily on corpora, which are usually restricted to a
few application domains. External linguistic tools and repositories (which
consist of databases of named entities, dictionaries, thesauri and general purpose
searchers like Google) are a good alternative. Regarding the nature of the
information to extract, only a few methods [
A schema of the approach we propose for Semantic Web population is shown in Figure 1. The information stored in natural language has to be extracted following strategies similar to those employed in information extraction systems. As a first step, a (web) document is parsed by a wrapper in order to determine its syntactical structure (chapters, sections, paragraphs, etc.). At this stage, each text segment is associated to a scope definition, that indicates which other segments can be related to it according to the hierarchical document structure. The focus of this paper is on the following step, the instance extractor web service. The extractor makes use of OWL ontologies, which define concepts and relations between them. We assume the existence of lexicons, which can be expressed in OWL language, and describe lexical rules to identify concepts and relations in the ontologies. Considering such lexicons, a parsed document can be processed to extract the ontology entities mentioned in the document, by using similarity functions between text fragments and lexical descriptions.
Extracted entities are used to define an initial instance set. Then, by applying several inference rules (which take into account the knowledge contained in the ontology) and the segment scope definitions, new relations can be added to connect instances in the initial set, and instances representing a unique object are properly joined. This process is performed non monotonically, as new instances are formed but others are deleted from the knowledge base in case of contradiction. As a result, the system generates a set of complex instances that semantically describe the whole content of the document, according to the given domain ontology. This approach has the advantage of being independent from the ontology representation formalism (e.g. frames versus Description Logic).
A formal definition of our proposal for the process of instance extraction is provided in the next sections. In Section 4, the basic definitions concerning ontologies, lexicon, instances and operations over relations defined on ontologies are given. Operations for joining and aggregating instances and the transformations these operations imply over all the knowledge base are described in Section 5.
Description Logic is considered as the logical framework that better adapts to the
requirements of the Semantic Web, and OWL has been defined as the language
for implementing ontologies. Different reasoners, which read the specifications
in OWL format and infer the complete meaning of concepts, have been
implemented. Any kind of reasoner (weak or strong, according to the set of tested
semantic restrictions) allows the recovering of two kinds of relation between
concepts: hierarchical and aggregational ones . This is the minimum useful
information needed for an information extraction process. Here, we assume the
availability of a reasoner to extract this essential information. An extended Tableau
algorithm for extract such information for ontologies in SHOIQ(D) languages is
described in [
The basic definitions required by our framework for operating over instances
are introduced next. An ontology (Definition 4) is defined by: a) the semantic
specification of a set of concepts and their relations, the abstract ontology
(Definition 1); and b) the description of the valid data for each concept, the lexicon
(Definition 3). These definitions are adapted from the work of Maedche [
Operations of specialization and abstraction over a relation of an ontology (Definition 2) allow to move through the relation hierarchy, recognizing the most specific relation associated to a given concept and the initial relation itself. In this way, relations R|c associated to a concept c can be derived. An instance (Definition 5) is defined by the set of relations and data associated to it. Definition 1. (Abstract ontology). An abstract ontology is a structure O= (C, ≤C , R, σ, card, ≤R, IR) consisting of: – two disjoint sets C and R whose elements are called concepts and relations, respectively, – a partial order ≤C on C, called concept hierarchy or taxonomy, – a function σ : R → C × 2C , called signature, – a cardinality function, card : R → N 0 × N , that represents the minimum and maximum cardinality of each relation, where N 0 and N are the sets of natural numbers including or not zero, respectively, – a partial order ≤R on R, called relation hierarchy, where r ≤R r′ implies that Q1(σ(r)) ≤C Q1(σ(r′)), for r, r′ ∈ R, The function dom : R → C with dom(r) = Q1(σ(r)) gives the domain of r, the function range : R → 2C with range(r) = Q2(σ(r)) gives its range. We call the set of datatypes T C = {c/c ∈ C, ∀c′, c′ ≤C c, ¬∃r ∈ R, dom(r) = c′}, T C ⊆ C.
As an example, the central fragment of the archaeology ontology which will be used in the rest of the paper to explain concepts and experimental tests is represented in Figure 2.
located name
Location
String has_Stratigraphy soil material dating Working Area Stratigraphy has levels
PartOf Zone
Lithic point Plainview found Artifact
Pottery bottle pot WA_ID BB_ID
Arch. Site has ID
PartOf
Level bed ID Bone Bed
flake Clovis
Material
depth belongs to
Period
Culture Bone
Date Range
Measure Paleolithic Neolithic
PaleoIndian Definition 2. (Specialization of a relation) We define the specialization of a relation r with respect to a concept c′, c′ ≤C dom(r), denoted with e |c′ (r), the relation: e |c′ (r) = r′|r′ ≤R r, c′ ≤C dom(r′) ≤C dom(r), ¬∃!r′′, r′′ <R r′ ≤R r, c′ ≤C dom(r′′) ≤C dom(r′) ≤C dom(r)
The specialization of a relation r with respect to a concept c′, called r′, is the unique and most specific relation for which the domain of r is an abstract concept of the domain of r′, and the latter is an abstract concept of c′. Thanks to the reflexivity in partial orders, r′ is relation r itself if no other more specific relations of r satisfy the conditions.
For example, given the ontology of Figure 2, the following is a specialization of a relation: e |Bone Bed (W orking Area.id) = Bone Bed.BB id
In this way, we can define the set of relations associated to a class c as the set: ∪c≤Cc′ {e |c (r)|dom(r) = c′}, which is called relation set of c and it is denoted as R|c . For example, R|Bone Bed = {Bone Bed.BB id, W orking Area.partOf }.
The abstraction of a relation r, a |c′ (r), can be defined in a similar way: a |Level (Bone Bed.BB id) = W orking Area.id.
It is easy to prove that r ∈ R|c, r′ ∈ R|c′ ⇒ (r′ = e|c′ (r) ⇔ r = a|c(r′)).
In order to bridge an abstract ontology with its related lexical terms, we introduce the concept of lexicon.
Definition 3. (Lexicon). A lexicon for an abstract ontology O is the structure L= (SC , SR, RefC , RefR, Insts) consisting of: – two sets SC and SR whose elements are called signs (lexical entities with specific semantics) for concepts and relations, respectively, – two relations RefC ⊆ SC × C, RefR ⊆ SR × R called lexical reference assignment for concepts and relations, respectively, – a set of instances, Insts : Literal → C ×2Literal×R×SR∪Literal, where Literal denotes the set of possible strings that can be used to nominate instances (see Definition 5).
RefC (s) = {c ∈ C|(s, c) ∈ RefC }, RefC−1(c) = {s ∈ Sc|(s, c) ∈ RefC }, RefR(s) and RefR−1(r) are analogously defined and,
AllRefC−1 = ∪c′≤CcRefC−1(c′) and AllRefR−1 = ∪r′≤RrRefR−1(r′)
In most cases, SC and SR are intentionally defined (i.e. through logical clauses); for example, “BoneBed1′′ ∈ RefC−1(BB id), “South of T exas′′ ∈ RefC−1(Location) and “situated′′ ∈ RefR−1(Arch Site.located).
Definition 4. (Ontology) A (concrete) ontology is a pair (O, L) where O is an abstract ontology and L a lexicon for O.
From now on, all definitions refer to the concept of concrete ontology. Besides, we will assume the existence of the function N ewLit that returns a new literal, not used on Insts, that can be used to nominate a new instance. Definition 5. (Instance) We define an instance named o of class c ∈ C − T C or simply instance, the set I|co = {(o, r, o′)|r ∈ R|c}. Function Inst(o) = (c, I|co) defines the relation between the name of an instance and its most specific concept and description. We call dc(o) the direct class associated to o: dc(o) = Q1(Inst(o)) = c. If I|co = ∅, then o is an empty3 instance of class c.
Using the previous example, the following sets are examples of instances: I|oA1rch site = {(o1, Arch site.name, “Bonf ire Shelter′′),
(o1, Arch site.located, “South of T exas′′)} I|oB2one Bed = {(o2, Bone Bed.BB id, “Bone Bed 1′′),
(o2, W orking Area.partOf, o1)} I|oS3tratigraphy = {(o3, Stratigraphy.partOf, o2),
(o3, Stratigraphy.f ound, o4), (o3, Stratigraphy.f ound, o5)} I|oB4one = ∅
I|oC5lovis = {(o5, Artif act.belongsT o, o6)} I|op6aleolithic = ∅
3An empty instance is an anonymous instance of which nothing is known.
In knowledge modeling, operations between concepts (or classes) like union, intersection, difference, complement, etc. are formally defined. However, this is not the case for the same set of operations between instances. In the Logic paradigm the description of all axioms associated with an instance is enough to represent it semantically, and a reasoning process has to be performed in order to retrieve a complete and compact description of an instance. In the Object-Oriented (OO) paradigm, the independence of instances based on their identifiers (name, physical direction, etc.) is assumed. Regarding the Logic paradigm, the proposed instance operations allow to maintain in the knowledge base a complete and compact description of all instances, avoiding repetitive simple reasoning for those cases in which the ontology can capture the structure of all possible instances (for example in most Description Logics Languages). For what concerns the OO paradigm, the proposal gives a formal definition for instance transformation.
In this section we introduce a set of definitions for transforming ontological instances of a concrete ontology through specialization, abstraction or combination operations, since they are the relevant ones for the present paper. These operations are used after concepts and relations mentioned in texts have been identified in order to construct complex instances (see Section 6). Definition 6. (Specialization of an instance) We call specialization of ′instance I|co into class c′, c′ ≤C c, denoted with I|co→→c(N′ewLit=o′), the instance I|co′ defined by the set: ′′ {(o′, r′, x′)|(o, r, x) ∈ I|co, e|c′ (r) = r′, range(r′) = {c′′}, ∃I|dxc→(x()N→ewc′L′it=x′) = I|cx′ } ∪ ∪ {(o′, r′, x)|(o, r, x) ∈ I|co, e|c′ (r) = r′, ∃c′′ ∈ range(r′), dc(x) ∈ T C, x ∈ SC (c′′)}.
The specialization of an instance is the replacement of each description (o, r, x) of o with the specialized description (o′, r′, x′), where r′ is the specialized relation of r respect to c′ and x′ the specialized instance of x according to the range of r′. Notice that o → (N ewLit = o′) renames the instance I|co with the ne w′ literal o′ obtained by using N ewLit. Additionally, if I|co is an empty instance, I|co′ will be an empty instance too, the specialization process only modifies its name and the actual concept associated with it.
For example, the specialization of the instance: I|oW orking Area = {(o, W orking Area.id, “Bone Bed 1”),
(o, W orking Area.partOf, o1)} to the class Bone Bed is: I|BBbon1e Bed = I|oW→oBrkbin1g Area→Bone Bed = {(Bb 1, Bone Bed.BB id, “Bone Bed 1′′), (Bb 1, W orking Area.partOf, o1)}.
Definition 7. (Specialization of an instance without missing information) We call specialization of instance I|co to class c′ without missing information, c′ ≤C c, denoted by I|co →→ˆˆ c(N′ewLit=o′) the instance:
I|co′ = I|co→→c(N′ewLit=o′) ∪ {(o′, r, x)|(o, r, x) ∈ I|co, e|c′ (r) = r′, |range(r′)| > 1}.
′
This definition includes also those descriptions whose relations can not be specialized by a unique concept. Definitions concerning abstraction of an instance with or without missing information can be similarly derived. cd represents the pairs of descriptions that should be unified in one value, employing the U n operation. In this way, the union of two instances is a new instance whose description is specialized to the most specific concept (between the joined instances), and where the pairs of functional relations are joined recursively by using the same union definition.
For example, instances: I|oArch Site = {(o, Arch Site.name, “Bonf ire Shelter′′)} and I|oA′rch Site = {(o, Arch Site.located, “South of T exas′′)} can be joined in instance:
I|oA′r′ch Site = {(o′′, Arch Site.name, “Bonf ire Shelter′′)},
(o′′, Arch Site.located, “South of T exas′′)} supposing that o′′ is the new literal returned by N ewLit function.
The last operation to formalize is the aggregation between instances. Its goal is to connect instances by using a set of relations and concepts semantically linked. For this reason, the notion of path between concepts is first introduced. Definition 9. (Path between concepts) The list (r1, c1), ..., (rn, cn) is a path of an ontology O if: – ∀ck, k ∈ {1, ..., n}, ck ≤C c∗, c∗ ∈ range(rk) – ∀ck, k ∈ {1, ..., n − 1}, ck ≤C dom(rk+1)
A path of an ontology is an ordered list of pairs (relation, concept) through which a concept (cn) can be reached from an other concept (dom(r1)) by using the definitions of the ontology.
For example, p = (Arch site.has Stratigraphy, Stratigraphy), (Stratigraphy.f ound, bottle) is a path between archaeological site and artifacts concepts, whereas p1 = (Bone Bed.BB id, BB id) is a path between Bone Bed concept and the label used by archaeologists to indicate the exact place. ′ Definition 10. (Aggregation of instances) We call aggregation of instance I|co′ to instance I|co, through path p = (r1, c1), ..., (rn, cn), c0 = dom(r1) ≤C c, cn ≤C {(o′′, r′, x) ∈ I′ |co→→co0′′ |r′ 6= r}∪ {(o′′, r1, x1)|∃I|cx1′1 = I|dxc(x) ∪ˆI|cx11 } {(o′′, r′, x) ∈ I|co→→co0′′ } ∪ {(o′′, r1, x1)} if Q2 card(r1) = 1, ∃(o, r, x) ∈ I|co, r1 ≤R r,
Q2 card(r) = 1, ∃I|dxc(x)∪ˆI|cx11 if Q2 card(r1) > 1, ∀r, r1 ≤R r where ∃(o′′, r′, x) ∈ I|co→→co0′′ , |{(o′′, r∗, v)|r∗ ≤R r}| < Q2 card(r) and where x1, x2, ..., xn are new instances described by: Inst(xi) = (ci, {(xi, ri, xi+1)}), i ∈ 1, ..., n − 1, ′ Inst(xn) = (cn, {(xn, rn, o∗)}) and I|con∗ = I|co′→→con∗ .
For example, instance I|boo′ttle = ∅ can be aggregated to I|oArch Site = ∅ by using path p of the previous example. I|oA′c′rh Site = {(o′′, has Stratigraphy, os)}, I|oSstratigraphy = {(os, Stratigraphy.f ound, o∗)} and I|boo∗ttle = I|boo′→ttloe∗ are new instances generated by the aggregation process.
Finally, the following two definitions establish the conditions to guarantee the consistence of the union and aggregation processes in a concrete ontology. ′ Definition 11. (Complementary instances) Two instances I|co and I|co′ of classes ′ c and c′ respectively, c′ ≤C c, are complementary, denoted with I|co ◦ I|co′ , if they satisfy at least one of the following conditions:
′ – at least one of instances I|co or I|co′ is empty,
′ ′ – for all relations in I|co →→ˆˆco′ ∪ˆI|co′ , i.e. the instance union of instances I|co and ′ I|co′ specialized into c′ without missing information, cardinality restrictions imposed by the ontology (through card function) are satisfied.
Two non-empty instances are complementary if their union, considering their non missing information specialization, maintains the cardinality property for all its relations. Notice that their union only contains the relations in R|c′ .
For example, I|oArch Site = {(o, Arch Site.name, “Bonf ire Shelter′′)} and I|Ao′rch Site = {(o, Arch Site.located, “South of T exas′′)} are complementary, and can be joined as in the previous example. However, I|oArch Site is not complementary to I|Ao′r′ch Site = {(o′′, Arch Site.name, “Bolomor Cove′′)}. In this case relation Arch Site.name is biyective (so, its maximal cardinality is one). Instances I|oB4one and I|oC5olvis are also not complementary because their concepts can not be ordered through the ≤C relationship. ′ Definition 12. (Aggregable instances) We say that instance I|co′ is aggregable ′∗ to I|co if one and only one path, p, exists between concepts c∗ and c , c∗ ≤C c ′∗ ′ and c ≤C c′ and I|co ←p I|co′ can be computed.
For example, instance I|oA1rch Site can be aggregated to instance I|oStratigraphy = {(o, Stratigraphy.f ound, o4), (o, Stratigraphy.f ound, o5)}, producing: I|oStratigraphy = I|oStratigraphy ∪ {(o, Stratigraphy.P artOf, o7)} where I|oW7 orking Area = {(o7, W orking Area.P artOf, o1)}
Definitions 11 and 12 allow us to introduce the following transformation rules of a concrete ontology , which define the transformations on the set of Insts functions during the union and aggregation operations. Given a concrete ontology, (O, L), it can be transformed to (O, L′) by the following: TR-Union t′o join two instances o, o′ ∈ Insts if I|co ◦ I|co′ then L′ = (SC , SR, RefC , RefR, Insts′), where Insts′ = Insts − {(o, dc(o), I|odc(o)), (o′, dc(o′), I|od′c(o′))} ∪ {(o′′, c′, I|co ∪ˆI|co′ )} ′ and o′′ is the new literal generated during the union process.
else prevent Contradiction.
TR-Aggreg to aggregate o′ to o, o, o′ ∈ Insts if o′ is aggregable to o then L = (SC , SR, RefC , RefR, Insts′), where ′ Insts′ = Insts − {(o, dc(o), I|odc(o)), (o′, dc(o′), I|od′c(o′))} ∪ {(o′′, c′, I|co ←p I|co′ ), ′ (o∗, dc(o∗), I|co′→→doc(o∗)), (xi, dc(xi), I|dxci(xi))|i ∈ {1, ..., n}}, p is the unique ′
∗ aggregation path from concept c to c′ ; o′′, x1, ..., xn are the new literals generated during the aggregation process as defined in Definition 10. else prevent Contradiction.
The TR-Union and TR-Aggreg transformation rules allow to perform a non monotonic processing on the knowledge base. With prevent Contradiction the caller process verifies if the transformation results are as expected. Given a pair of instances of the Insts set (the knowledge base) which have to be joined or aggregated, the caller process could delete both of them in case of contradiction, and/or adequately update the knowledge base. Therefore, incremental grow of the database is not guaranteed, but rather depends on the restrictions defined by the ontology used during the processes of union and aggregation. The whole instance extraction process, which uses the above rules to generate the set of complex instances described in a text, is explained next. 6
An initial instance set has to be obtained from a text before the transformation
rules can be applied. Therefore, the text has to be analyzed and the fragment
texts related to ontological entities have to be extracted. The solution we propose
in [
We consider texts with a syntactical division in chapters, sections, subsections, etc., and such division defines the scope (initial and final paragraphs)
Section 1 Section 2 Section 1.1 Section 1.2
Logical structure Partition’s scopes
T P T P T P P P T P P P T P P P Text partitions of the subject of each particular part (as represented in Figure 3). Hence, the descriptive fragment texts (i.e. those fragments that describe ontology entities) are used to form the initial set of instances and each of these instances is also associated to the semantic scope of the text partition where the entity has been found. The scope of an instance delimits the instances that can be joined or aggregated with it: those whose scope overlap with it. Then, instances in the initial set are combined following the concepts for joining and aggregating instances introduced in Sections 4 and 5.
Algorithm 1 formally describes the process of complex instance extraction. The first step, the construction of the initial set of instances, considers all recognized entities and define a new instance for each of them: if the recognized entity is a concept or a relation and it is possible to infer an unique class having this relation, a new instance of the (inferred) concept is created and a scope is properly associated to it. During the second step, instances whose scope are overlapped are combined in the order in which these instances appears, while no contradictions are obtained with the application of the TR-Union and TRAggreg rules. The final result of all this process is a set of complex instances representing the information related to the ontology which is contained in the text. 7
In spite of the importance and the number of different approaches dedicated to information extraction, no universally recognized test datasets are available for comparing the results of different systems. Available test data are reduced to entities, or to very simple concepts with few levels of aggregability. These datasets are of very little use in this case, as our work is especially focused on the capability of using relevant entities to create complex instances, selecting in each case appropriate pair of instances to be joined or aggregated. Hence, until datasets suitable for comparison become available, we have to test our approach on custom test data. We present here the experimental results obtained applying our approach to a set of paragraphs extracted randomly from nine archaeological memories written in Spanish.
Figure 4 shows a screenshot of our information extractor system. A paragraph on an archaeological site memory has been selected, and the complex instances Algorithm 1 Complex instance extraction Require: (O, L), T {(O, L), ontology} {T , text of the ontology domain.} Ensure: Insts′ {Insts′, a set of new instances extracted from T .} Let EO be the ontological entities recognized in the text.
Construction of initial set of instances. cmbInsts = ∅; Insts′ = ∅ for all e ∈ EO do o ← N U LL if e ∈ C (if e is a concept) then
o ← N ewLit; Insts′ ← Inst′ ∪ {(o, e, ∅)} else if ∃!p, path(c, c′) that contains the relation e (if e is a relation and exists a unique path that contain it) then o ← N ewLit; Inst′ ← Inst′∪{(o, c, ∅)}; o′ ← N ewLit; Inst′ ← Inst′∪{(o′, c′, ∅)} Apply TR-Aggreg to aggregate o′ to o if o 6= N U LL then cmbInsts = cmbInsts ∪ {o} Associate to o the scope of e (as explained in the Section 6, the scope is defined considering the syntactical division of the text and the position where e has been found.) Combining entities. for all {o} ∈ cmbInsts do if o has not been joined neither aggregated to other instance then for all {o′} ∈ cmbInsts after e, where o and o′ scopes are overlapped do if dc(o) ≤C dc(o′) ∨ dc(o′) ≤C dc(o) then
Apply TR-Union to unify o and o′ else
Apply TR-Aggreg to aggregate o to o′ or o′ to o
Interchange o and o′ if o′ has been aggregated to o. if Contradiction has been produced in this iteration then
break (o can not be extended any more) extracted from it are shown and highlighted on the text, and transcribed in the main window. In this example, the complexity of the process and the capacity of the system to infer instances with a high aggregations depth (in this example, five levels of aggregation have been inferred) can be appreciated.
The ontology is composed by 194 classes and 131 relations. It is not very large,
but complex enough in order to obtain non-trivial specialization and aggregation
depths (nine and five levels respectively). Current semantic annotation tools
usually deal with much simpler ontologies. The lexicon is composed of synonyms
and contexts for the concepts and relations of the ontology. Besides, a set of
28 named entities (e.g. archaeological sites and stratigraphic unit names, weight
and dimensions of materials, percentages, etc.) were defined and the EEON
(acronym of named entities extractor, in Spanish) module [
A characterization of the sample texts, according to the number of paragraphs, instances and relations in them, as well as the number of correct and incorrect extractions obtained by the system, the precision and recall for each sample text and for the overall dataset are shown in Table 1. The results allow to compare the correctness of: 1) the class associated to each part, 2) the relations associated to each identified instance, and 3) both elements in complex instances.A retrieved complex instance is correct if it belongs to the same class and describes an overlapped segment text of a target complex instance. A retrieved relation is correct if it belongs to a correct instance and the same relation (name and associated value) appears in the corresponding target instance.
The text samples contain 96% of the ontology entities, and a precision of around 99% was achieved in the extraction of these entities. This good result constitutes the ideal starting point for the manipulation and combination of initial instances into more complex ones.
All precision measures obtained for complex instances are over 90%, suggesting that the great majority of extracted instances are correct. It is important to 4Precision and recall are classical measures of Information Retrieval Systems. Precision is associated with the capacity of extracting entities correctly, while the recall is associated with the capacity of extracting entities as much as posible. highligth that they are complex instances: this implies that not only the lexical definitions and the disambiguation process allow to retrieve the correct entities, but also that aggregation operations are correctly performed on the right instances. On the other hand, at least 75% of all complex instances in a sample text were correctly extracted. This relatively low value of recall is due to the use of lists and/or very complex explanations to describe instance sets. This kind of expressions lead the system to wrong decisions, creating a single complex instance instead of several ones. This error decreases the recall values for both instances and relations (see results for texts 2 and 6), and the same problem affects relation extraction precision, as can be observed for texts 5 and 7.
The overall precision of the system is around 93%, with a recall of 82%. These
results are very satisfactory and promising in comparison with other systems
available in the literature [
Summarizing, our proposal provides good results in complex situations, but it is highly dependent on the quality of the entity recognition process, that is, the quality of the lexicon, and how well the text fragments can be associated to the correct ontological entities. In restricted knowledge domains (like archeology, bioinformatics, etc.) the use of the appropriate entity recognizers and a good, controlled vocabulary allow to discover complex instances with high values of precision and recall. On the other hand, in open domains a syntactic and semantic analysis of the texts might be required. 8
Information extraction systems are more and more essential for maintaining, using and interchanging information, especially in huge, unstructured and evergrowing environments such as the web. We presented here a novel system to extract complex ontological instances and relations. Experimental results, although not exhaustive yet, are very promising, and the system design considers some critical issues such as OWL-awareness and scalability, that make it a useful tool for populating and updating data in the Semantic Web. The approach is based on non monotonical processing, and makes use of an ontology and entity recognizers and disambiguators in order to combine adequately an initial set of instances. Exhaustive analysis and experimentation of the proposal is being performed in a variety of application scenarios.