The paper presents the framework for conceptual modeling which has been used in on-going project of developing fact extraction technology on textual data. The modeling technique combines the usage of conceptual graphs and Formal Concept Analysis. Conceptual graphs serve as semantic models of text sentences and the data source for formal context of concept lattice. Several ways of creating formal contexts on a set of conceptual graphs have been investigated and resulting solution is proposed. It is based on the analysis of the use cases of semantic roles applied in conceptual graphs and their structural patterns. Concept lattice building on textual data is interpreted as storage of facts which can be extracted by using navigation in the lattice and interpretation its concepts and hierarchical links between them. Experimental investigation of the modeling technique was performed on the annotated textual corpus consisted of descriptions of biotopes of bacteria.
Conceptual modeling in the Natural Language Processing (NLP) is a way of modeling
semantics. Semantics of texts is transformed to semantics of conceptual models at a
high level of abstraction, in terms of concepts. Conceptual graphs (CGs) [
Another paradigm of conceptual modeling is Formal Concept Analysis [
The idea of joining two paradigms of conceptual modeling - conceptual graphs and
concept lattices - seems very attractive but not elaborated in the FCA community.
There are several its realizations due the years, from early implementation in [
This idea may get a second breath when FCA is utilized on textual data and
conceptual graphs serve as conceptual model of text semantics. Acquiring conceptual
graphs from natural language texts is non-trivial problem but it is quite solvable [
The main problem which arises in CGs – FCA applications is the problem of
building formal concepts on conceptual graphs. Solution of this problem and the
whole principle of applying FCA on textual data are closely depended on the real-life
problems have been solved with FCA on textual data [
One of the fields where Text Mining applications are growing rapidly is
Bioinformatics. New term of Biomedical Natural Language Processing (BioNLP) has
been appeared there [
In this paper we present the framework for conceptual modeling which has been used in on-going project of developing fact extraction technology on textual data.
The next section of the paper contains brief description of FCA basics and conceptual modeling technique which is used in the framework.
Section 3 is devoted to the framework; its structure and functionality are described there.
In the section 4 current experimental results of using framework on bacteria biotope textual corpus are presented and section 5 contains conclusion and planning future works. 2
We are developing conceptual modeling technique which combines the usage of conceptual graphs and conceptual lattices from Formal Concept Analysis. Consider some FCA basics needed for understanding the modeling technique.
There are two basic notions FCA deals with: formal context and concept lattice.
Formal context is a triple K == (G, M , I ), where G is a set of objects, M – set of their
attributes, I ⊆ G × M – binary relation which represents facts of belonging attributes
to objects. The sets G and M are partially ordered by relations ф and – ,
correspondingly: G == (G, ф) , M = (M ,– ) . Formal context may be represented by
[
A formal concept is a pair (A, B) of subsets of objects and attributes which are connected so that every object in A has every attribute in B, for every object in G that is not in A, there is an attribute in B that the object does not have and for every attribute in M that is not in B, there is an object in A that does not have that attribute.
The partial orders established by relations ф and – on the set G and M induce a partial order ≤ on the set of formal concepts. If for formal concepts (A1, B1) and (A2, B2), A1 ф A2 and B2 – B1 then (A1, B1) ≤ (A2, B2) and formal concept (A1, B1) is less general than (A2, B2). This order is represented by concept lattice. A lattice consists of a partially ordered set in which every two elements have a unique supremum (also called a least upper bound or join) and a unique infimum (also called a greatest lower bound or meet).
According to the central theorem of FCA [
The main problem in applying FCA on textual data is the problem of building formal context. If textual data is represented as natural language texts then this problem becomes especially important.
There are several approaches to the construction of formal contexts on the textual
data, presented as separate documents, as data corpora. One, mostly applied variant
of context is that its objects are text documents and the attributes are the terms in
these documents [
Another variant of formal context is building directly on the texts. In the general case, various word combinations constitute its concepts and the number of such concepts may be very large. An advantage of such variant is that this context contains potentially more information about texts than previous one and more general problems such as fact extraction problem can be solved on that formal context. The disadvantage of it is its great dimension and possible many pointless concepts.
Restricting the dimension of formal context and giving it more semantics is doing
by representing in it the various features of its source texts: semantic relations
(synonymy, hyponymy, hypernymy) in a set of words for semantic matching [
For building formal context, one needs to distinguish some of these lexical
elements in texts as objects and attributes. There are following approaches to solve
this problem:
• adding special descriptions to texts which mark objects and attributes and partial
order – this is usually done manually;
• using semantic models of texts and corpus tagging [
We apply the second approach and use conceptual graphs for representing semantics of individual sentences of a text. 2.3
1. Acquiring a set of conceptual graphs from input texts. Conceptual graph [
2. Aggregating the set of conceptual graphs. Aggregation is needed to exclude excessive dimension of conceptual models, not related to useful information. We have tested two ways of conceptual graphs aggregation: conceptual graphs clustering and restricting the number of conceptual graphs by identifying and excluding sentences which are not corresponded to the problem solving with the current technique. 2 The lightweight online version of CGs Maker for simple English and Russian texts can be found at http://85.142.138.156:8888 . 3. Creating formal contexts. One or several formal contexts are built on the aggregated conceptual graphs. The number of formal concepts and the method of building them have been determined in the solving problem.
4. Building concept lattice. Having a concept lattice, it is possible to identify connections between the concepts according to the principle of "common – particular". Each concept, the node in the lattice is interpreted as the set of potential facts of certain level, which is associated with other facts.
5. Fact extraction from concept lattice. Concept lattice is the data storage for fact extraction system. This system has domain oriented user interface for query processing and generating output.
This paper reflects results of investigations corresponded to steps 1-3 of the
process. On the step 4 we used standard open source tool for building and visualizing
concept lattices [
The crucial step in the described process of CGs – FCA modeling is creating
formal contexts on the set of conceptual graphs. At first glance, this problem has
simple solution: those concepts which are connected by "attribute" relation have been
put into formal context as its objects and attributes. Actually the solution is much
more complex. To illustrate it consider conceptual graph for the sentence “Xylella
fastidiosa is a gram-negative fastidious, xylem-limited bacterium” shown on Fig. 1.
This sentence is from bacteria biotopes textual corpus [
Conceptual graph on the Fig.1 has four conceptual relations “attribute” but only three of them indicate real objects and attributes for formal context. Using “fastidiosa” as object and “Xylella” as attribute in the formal context is wrong way because “Xylella fastidiosa” is known full name of this bacterium. Full names of bacteria have to be objects in the formal context devoted to bacteria. Word combinations denoting the names of bacteria must be recognized before conceptual graphs building. There is no other way of doing this than to use an external source of information, for example, the corpus tagging.
We also realize the following rules for creating formal contexts on conceptual graphs. 1. Not only individual concepts and relations, but also patterns of connections between concepts in conceptual graphs represented as subgraphs have been analyzed and processed. The pattern “agent - patient” is mostly frequent in biotope texts. 2. The hierarchy of conceptual relations in conceptual graphs is fixed and taken into account when creating formal context. Such hierarchy exists on the Fig.1: relations “agent” and “patient” are on the top level and relations "attribute" belong to underlying level. Using this hierarchy of conceptual relations we can select for formal contexts more or less details from conceptual graphs. This makes conceptual graphs more power and flexible semantic model for FCA than n-grams or collocations. 3. FCA – model for fact extraction is domain specific. Domain information is also taken into account in conceptual graphs building. This information is from external resources – thesauruses or tagging of textual corpuses.
Concrete implementations of these rules are in the section 4. 3
Architecture of the CGs – FCA modeling framework is shown on the Fig. 2. Consider its main elements.
Database. Database is very important part of the framework. We use relational
database on the SAP-Sybase platform. It was built with CASE technology
PowerDesigner™ [
Conceptual graphs building module. This module and several other modules constitute the NLP block of modules of the framework. They realize our algorithm of acquiring conceptual graphs from texts, visualization of conceptual graphs and their clusters, interaction with external resources including WordNet.
English and Russian languages have been supported in the framework. The framework has internal dictionaries and may communicate with external ones.
Representing of modeling results. Modeling results have been presented as visualization of conceptual graphs and concept lattices as in table and textual forms. Storing all objects in database allows analyzing its data and computing conceptual graphs and concept lattice characteristics.
Programming environment. Java is the main programming platform which is used in the framework. Some modules of NLP block have been written on PowerScript language of SAP-Sybase platform.
Experimental evaluation of CGs – FCA modeling technique has been carried out
on the textual corpus of bacteria biotopes which is used in the innovation named as
BioNLP Shared Task [
Biotope is an area of uniform environmental conditions providing a living place for
plants, animals or any living organism. According to [
It is preliminarily clear that the task of extracting the names of bacteria and the task of extracting locations and pathogenicity have different complexities. For extracting the names of bacteria some words or collocations (Xylella fastidiosa) have to be analyzed in the text. Locations and pathogenicity may be represented by more complex and long word combinations. As for bacterium Xylella fastidiosa on the Fig. 1, the example of its location is the following fragment from the text about it: “the bacteria … receive a safe environment and metabolites from the insect”. To extract “insect” as location of bacterium we need to analyze some relations between words in the sentence. This is done also through the use of conceptual graphs.
Biotope texts tagging includes full names of bacteria, its abbreviated names and unified key codes in the database. We add additional tags if special words (extreme, obligately, etc.) recognized in the texts.
A BioNLP data is always domain-specific. All the texts in the corpus [
Our solution of the task of Named Entity Recognition is supported by conceptual graphs. As it is illustrated above (Fig. 1) conceptual graphs can represent names of bacteria as named entities. Named Entity Recognition also includes anaphora resolution. 4.1
Anaphora resolution is the problem of resolving references to earlier or later items in the text. These items are usually noun phrases representing objects called referents but can also be verb phrases, whole sentences or paragraphs. Anaphora resolution is the standard problem in NLP.
Biotope texts we work with contain several types of anaphora: • hyperonym definite expressions (“bacterium” - “organism”, “cell” - “bacterium”), • higher level taxa often preceded by a demonstrative determinant (“this bacteria”, “this organism”), • sortal anaphors (“genus”, “species”, “strain”).
For anaphora detection and resolution we use a pattern-based approach. It is based on fixing anaphora items in texts and establishing relations between these items and the objects in conceptual models we use. These objects are bacteria names for “Entity” context, mentions of water, soil and other environment parameters for “Areal” context and names and characteristics of diseases for “Pathogenicity” context.
Corpus tagging is also used for anaphora detection. In particular encoding bacteria (for instance bacterium Burkholderia phytofirmans is encoded as PsJN) is found from tagging and further used as its name in text processing.
Noise is constituted by the text elements that contain no facts or cannot be interpreted as facts. Also noise consider the data that are deliberately excluded from consideration, for example, information about when and by whom a bacterium was first identified. 4.2
We have selected 130 mostly known bacteria and processed corresponding corpus texts about them. Three formal contexts of “Entity”, “Areal” and “Pathogenicity” had built on the texts. They have the names of bacteria as objects and corresponding concepts from conceptual graphs as attributes.
Table 1 shows numerical characteristics of created contexts.
Context name
As it is followed from the table there is relatively small number of formal concepts in the contexts. This is due to the sparse form of all contexts generated by conceptual graphs and noise reduction. 4.3
Extracting facts from concept lattices is realized by forming special views constructed on the lattice and corresponded to certain property (intent in the lattice) or entity (extent in the lattice) on the set of bacteria. Every view is a sub lattice. It shows the links between concrete bacterium and its properties.
An example of such view as the fragment of lattice is shown on Fig. 3. The lattice on the Fig. 3 contains formal concepts related to the following bacteria: Borrelia turicatae, Frankia, Legionella, Clamydophila, Thermoanaerobacter tengcongensis, Xanthomonas oryzae. Highlighted view on the figure corresponds to gram-negative property of bacteria. Such bacteria are resistant to conventional antibiotics.
Using this view, some facts about bacteria can be extracted: • only three bacteria from the set, Thermoanaerobacter tengcongensis,
Clamydophila and Xanthomonas oryzae, are gram-negative; • two gram-negative bacteria, Thermoanaerobacter tengcongensis and Xanthomonas oryzae, have the shape as rod; • one of gram-negative bacteria, Clamydophila, is obligately pathogenic. Note that attribute obligately pathogenic was formed directly from the same two words in the text according to the rule of marking words denoting extreme situation.
We can compare our results with the known ones, most completely presented in the
work [
This paper describes the first but very important stage of creating environment for performing experiments of CGs – FCA modeling in the project of creating fact extraction technology on natural language texts. Some parts of this project are under construction but current results demonstrate effectiveness of CGs – FCA modeling.
Conceptual graphs were recognized as valid low level conceptual model for creating high level such model of concept lattice. Using conceptual graphs, it is possible to control semantic depth of representing sentences in formal concepts by selecting certain levels (sub graphs) of graph structure.
Among the topics of our future work there are the following.
Now the verb-centric approach which we use in acquiring conceptual graphs is not
fully applied for creating formal contexts. When conceptual graph has the pattern
<concept> - (agent) – <verb> – (patient) - <concept> the verb serves as condition
which links two concepts. In other patterns with other conceptual relations including
attribute verbs play the same role. This opens the need to construct tricontexts on
conceptual graphs. We plan to construct multidimensional data model on our database
under SAP PowerDesigner™ CASE technology and apply OLAP for modeling
tricontexts and triclusters.
We also plan to use SAP HANA Environment [
Acknowledgments. The paper concerns the work which is partially supported by Russian Foundation of Basic Research, grant № 15-07-05507.