The paper considers the use of elements of Formal Concept Analysis - multidimensional or polyadic formal contexts - to extract information from natural language texts. We propose the method for constructing polyadic formal contexts by means of Semantic Role Labeling and Abstract Meaning Representation (AMR) of texts. Using semantic role labeling, a conceptual graph is created for each sentence of the text, and a specific scheme of abstract meaning representation of the sentence is developed based on its elements. The polyadic formal context is a multidimensional tensor, whose points are elements of an AMR scheme. To extract information from a polyadic formal context, data associations as sub-contexts of the original context are built. Each such subcontext is associated with a specific element of the AMR scheme. Queries to associations return responses that preserve the meaning of the phrases according to the AMR scheme. The method was tested in the task of finding dependencies between texts on the corpus of abstracts of scientific articles on biomedical subjects of the PubMed system.
The current state of the Computational Linguistics is characterized by the active involvement of mathematical methods of Data Analysis: methods of machine learning, algebraic methods, and methods of graph theory. Such synthesis is doubly useful. On the one hand, it allows in some cases to define objects used in computational linguistics in a new way and also to offer new solutions in Natural Language Processing. On the other hand, the applications of these methods in specific tasks enrich the methods themselves, opening up new areas of development in them. These observations prove to be true in relation to experimental material involved in our study. In it we apply the Formal Concept Analysis (FCA), a mathematically rigorous theory of conceptual modeling, and its main object, the formal context, which in some sense generalizes the concept of context in linguistics. In this paper we prove that clustering used in the Formal Concept Analysis (FCA-clustering) is ineffective in the tasks of extracting information from our specific formal contexts built on texts.
Copyright ©2020 for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
The paper proposes another approach to the clustering of formal context data, based on the construction of data associations with a specific AMR scheme. Information Extraction (IE) from data is effective when the data models used for this purpose are sufficiently informative by themselves. This is especially true for information extraction from natural language texts. To extract information from text data, a common scheme «model + resource» is used.
The model reflects the structure and parameters of the information retrieval target. Forms of models are various. This can be a lexical-grammatical template in a fact extraction problem, or a matrix or graph in procedures based on mathematical models. Linguistic resources are used to train models: text corpora, ontologies, and thesauri. The peculiarity of multidimensional formal contexts used in this work is that they can simultaneously be models of objects, for example, in thesauri development, and information resources in question answering systems. In this paper, formal contexts are constructed using an abstract meaning representation of the text. This ensures that they are informative as models: the semantics of AMR schemes are preserved in the model and in the query results it delivers.
The method developed in this paper is tested on the texts of the AGAC corpus
(Active Gene Annotation Corpus) which contains abstracts of scientific articles on
biomedical topics of the PubMed system [
Formal Concept Analysis (FCA) [
Briefly consider the main issues of the FCA. Classical FCA deals with two basic
notions: formal context and concept lattice. Formal context is a triple
K=(G, M , I ) where G is a set of objects, M – set of their attributes, I ⊆ G × M –
binary relation which represents facts of belonging attributes to objects. Formal
context may be represented by [
with the properties of A′ : ={m ∈ M |< g, m >∈ I for all g ∈ A} and B′ : ={g ∈ G |< g, m >∈ I for all m ∈ B} then the pair (A, B) that A′ =B, B′ =A is named as formal concept. The composition of mappings demonstrates following properties of A and B: A ''
=A, B '' =B; A and B is called the extent and the intent of a formal context K =(G, M , I ) respectively.
By other words, a formal concept is a pair (A, B) of subsets of objects and attributes which are connected so that every object in A has every attribute in B, for every object in G that is not in A, there is an attribute in B that the object does not have and for every attribute in M that is not in B, there is an object in A that does not have that attribute.
If for formal concepts (A1, B1) and (A2, B2), A1 A2 and B2 B1 then (A1, B1) ≤ (A2, B2) and formal concept (A1, B1) is less general than (A2, B2). This order is represented by concept lattice. A lattice consists of a partially ordered set in which every two elements have a unique supremum (also called a least upper bound or join) and a unique infimum (also called a greatest lower bound or meet). 1.1
First of all, it is necessary to define correlation between the notions of formal and linguistic contexts, the former being substantiated in algebraic theories, the latter being part of language representations.
Linguistic theories provide a variety of context types. In general, a context is
regarded as an obligatory condition for actualization of basic relations within a
language system, namely, syntagmatic and paradigmatic relations considered on
morphological, syntactic and semantic levels. On the one hand, there are approaches which
take into account the scope and size of linguistic contexts. This view is characteristic
for distributional semantics based on the assumption that semantic similarity of
lexical items arises from their contextual similarity. This assumption is well-grounded in
the works of L. Wittgenstein, Z. Harris, J. Firth, etc. who are considered to be the
founders of this trend in contemporary linguistics (cf. the survey [
Inspite of external differences, particular contexts considered in linguistic theories
can be generalized as instances of formal contexts. Let’s consider a certain class of
context relations described as verbal constructions, or valency frames [
Interpretability of the basic notions of FCA from linguistic point of view explains
its effectiveness in a wide range of applications in Text Mining [
Formal Concept Analysis may be defined as «the paradigm of conceptual modeling
which studies how objects can be hierarchically grouped together according to their
common attributes» [
Among the advanced issues of FCA there is the study of multidimensional formal
contexts which can be represented as n-ary relations R D1 D2 ... Dn on data
domains D1, D2 , ..., Dn . For n = 3 these domains have the meanings of «objects»,
«attributes» and «conditions» and FCA on formal contexts of this dimension has been
distinguished as Triadic Formal Concept Analysis [
An important result was obtained here, consisting in the fact that every threedimensional concept of a conceptual lattice belongs to some tricluster. According to multimodal clustering, for any dimension of formal context, the purpose of its processing is to find n-sets H X1, X 2 , ..., X n which have the closure property u ( x1, x2 ,..., xn ) X1, X 2 , ..., X n , u R , (1) j 1, 2,..., n, x j D j \ X j X1,..., X j {x j}, ..., X n does not satisfy (1). The sets H X1, X 2, ..., X n constitute multimodal clusters.
As two-dimensional biclusters are built formally, as for the dimension n ≥ 3 clustering is performed with the use of various measures of proximity. Accordingly, the problem of interpretation of multimodal clusters in the context of the selected proximity measure arises. 2
The central notion of Formal Concept Analysis, the notion of formal concept seems very attractive for applying it in the areas where the term «concept» is used naturally. Natural Language Processing (NLP) is just that area. The cherished goal in the NLP is computerized understanding of texts. One a way of such understanding is using concepts being acquired from texts. Formal contexts potentially contain concepts but it is evident that expressiveness of standard two-dimensional formal contexts is not enough for modeling all peculiarities of natural language texts. So we apply multidimensional or polyadic formal contexts constructed on texts.
Consider in general the process of constructing polyadic formal contexts on natural language texts. It includes the following steps.
Establishing the problems to solve. Determining the range of tasks that the developed model is oriented towards in the form of a multidimensional formal context. In a general setting, these are the tasks of extracting information. They come down to extracting named entities from the text, extracting relationships, facts, and events. These may be the results of a query to a system that uses formal contexts.
Choosing a semantic text model. There should be an intermediate link between the text and the formal context the link as semantic model which is the data source for the formal context. As such a model, we chose the abstract-semantic representation of the text. To construct AMR-schemes, conceptual graphs are used.
Formal context construction. At this stage, it is necessary to choose the dimension of the context, the composition of the sets, D1, D2 , ..., Dn and build the relation R D D2 ... D . The constructed multidimensional formal context should be 1 n implemented as a storage object, for example, in a database in such a way as to ensure work with context by executing queries to it.
We apply conceptual graphs (CGs) [
Certain problems arise when using conceptual graphs as input to formal contexts.
Among them there is the problem of redundancy of conceptual graphs. A conceptual
graph acquired from quite a long sentence may contain many various semantic roles,
and it is difficult to represent the variants of connections they specify in the formal
context, even when the dimension of a context is greater than two. The solution to this
problem is to aggregate conceptual graphs. An aggregated conceptual graph is a
smaller graph that summarizes the information contained in the original graph [
The method of aggregation that we apply is based on the construction of an Abstract Meaning Representation (AMR) on each conceptual graph.
AMR [
Let's call the AMR schema a template T (C, S) where C is a set of concepts, S is a set of semantic roles, they both are from conceptual graph. Concrete content of AMR schema is defined by certain values (meanings) of C and S and it has a certain meaning too. For example, the template T(C, S) = < Concept_1 > ← (“Agent”) ←<Verb>→(“Patient”) →<Concept_2> (2) specifies the AMR schema with the meaning «who did what to whom». The template (2) defines a conceptual graph in which «Agent» and «Patient» are the names of semantic roles, Concept_1, Concept_2 – words being its concepts, «Verb» is conceptverb from conceptual graph.
Figure 1 demonstrates an example of interpreting AMR schema as sub graph of conceptual graph.
Conceptual graph on the Fig. 1 derives AMR schema «who did what to whom» with the content «SHP-2 attenuates function» according with the template (2). This AMR schema represents the meaning of the whole sentence and, certainly, represents it very broadly. Using AMR schemes, semantic compression of text sentences is performed.
There are two propositions which we can formulate based on the analysis of works
[
The polyadic formal context is constructed as follows. By semantic role labeling for each sentence of the text, a conceptual graph is constructed, on the elements of which a concrete AMR scheme is created. The formal context K D D ... Dn is a 1 2 multidimensional tensor whose points are the elements of the AMR scheme for each representation, ki, j,...,n = {ci , c j ,..., cn} where ck, k = 1, 2,..., N are the concepts of the concept graphs, N is the total number of concepts obtained on the processed text. The number of points in the formal context matches the number of AMR schemata found in the text. The vast majority of points in a formal context are meaningful phrases, for example, the phrase «SHP-2 attenuates function» from figure 1 is a point <SHP-2, attach, function> in a three-dimensional context.
context, it is necessary to organize its storage and access to its content in order to solve the problems of extracting information. Information is extracted by querying a polyadic formal context.
There is the following idea concerned with Conceptual Modeling. If a query to a
conceptual model can be represented as an element of this model itself − for example,
as its concept, then the refinement of this query or even the answer to it is contained
in concepts adjacent to the concept-query. This idea also holds for concept lattices
and has been tested in several papers [
Although this cluster can be useful by demonstrating the relationship of objects from the first subset through words from the second and third subset, it is impossible to extract information from it, for example, about what exactly causes dysplasia. It turns out that individual elements of multidimensional formal context, its points, contain specific information in the form of an AMR scheme, but after processing the context this information is lost.
Thus, in order to extract information from multidimensional formal contexts based on AMR schemes, a different than FCA-clustering method is needed.
In our method, specific clusters − associations are built on formal contexts. An Association is a set of points ordered relative to the selected word position in the AMR scheme for a point. This corresponds to the logic of the AMR scheme: certain semantic elements are selected in it. The Association includes all words in the selected position of the AMR scheme. Therefore, an Association is a cluster built on the basis of the proximity measure «belong to a certain position of the AMR scheme». On the other hand, the Association is a function А(x1,…, xp) whose argument can be a given word or a set of p words belonging to the k-th position of the AMR scheme.
The meaning of highlighting such associations is closely related to the logic of queries to the formal context. These queries usually correspond to the structures of the AMR charts. 3 3.1
Let’s discuss an example of applying FCA to data analysis in a specific area. One of
the areas where NLP applications become more in demand is Bioinformatics. The
Biomedical Natural Language Processing (BioNLP) [
RE is another standard task of BioNLP. Relations are associations among biomedical entities. The simplest relations are binary, involving only the pair-wise associations between two entities. But biomedical relationships can involve more than just two entities. This kind of relationship is actual in the task of event extraction. In our time, named as genomic era, much of BioNLP work has focused on automatically extracting interactions between genes and proteins. Other associations include interactions between proteins and mutations, proteins and their binding sites, genes and diseases, genes and phenotypic context.
Leading BioNLP research groups are mainly interested in processing English data, although Russian biomedical texts attract growing attention.
Researchers collected and prepared for distribution a great amount of textual data. BioNLP competitions inspired creation of richly annotated corpora for NER, RE, Semantic Role Labeling (SRL), etc. The given empirical data is a great asset to computational linguists working in the field of Bioinformatics. 3.2
Experiments aimed at the empirical verification of our approach were carried out for
texts of the AGAC corpus (Active Gene Annotation Corpus) which contains abstracts
of scientific articles on biomedical topics of the PubMed system. The corpus was
created for BioNLP Shared Tasks 2019 competition [
The problem of finding the relationships of texts is well known in the field of IE and has a variety of options. In our experiments, we studied a variant in which it is not known in advance by what attributes the links between texts are established. These attributes, which are ultimately reduced to subsets of words, are determined by analyzing the contents of texts, which in this case are replaced by a formal context built on them.
In our experiments, we compared the informativeness of two formal contexts built on texts in accordance with 3 and 5 element AMR schemes. The three-element scheme has the form (2), and the five-element AMR scheme has the following form: An additional dimension was included in each context to fix the number of the text to which this point belongs. As a result, contexts of dimensions 4 and 6 were subject to processing.
Obviously, multi-element AMR schemata allow more detailed modeling of the semantics of a sentence. Formal contexts built on their basis are more informative. This position was checked in experiments. The experiments included the following steps. 1. Building associations on selected positions of the AMR-scheme of the formal context. 2. Generating queries for associations based on query words 3. Obtaining query results in the form of clusters containing formal context points. 4. Interpretation of clusters.
Consider some experimental results. Associations were created on both formal contexts regarding the position of the subject of the action − the first position for the three-element AMR scheme and the second position for the five-element one. Next, the size and content of each association were estimated.
Domain terms were highlighted in the corpus. The term «mutation» has extensive connections, it organizes one of the most voluminous associations. Indeed, most of the texts of the corpus are devoted to the study of various manifestations of mutation and its influence on organisms. Therefore, our queries to associations were performed using the keyword «mutation». The results of the query are clusters. The question that determines further actions with the resulting clusters is: «What does the mutation manifest itself on?» The implementation of this request on clusters was carried out by building associations with respect to the position of the action object − the third position for the three-element AMR scheme and the fourth position for the five-element one. The query words obtained in the constructed associations were compared with text numbers and then presented for analysis.
Responses to association requests are generated in tabular form. If the result of a two-element query to associations is presented as a cross-table, it is interpreted as a two-dimensional formal context. In this case, it can be visualized as a concept lattice according to the classical version of FCA.
Fig. 2 (a) shows the sub context as a cross-table of the four-dimensional formal context constructed for three-element AMR scheme (2), Fig. 2 (b) shows concept lattice. Fig. 2. The subcontext of the formal context built for the three-element AMR-scheme and its visualization in the form of concept lattice The query that generates the result in Fig. 2, can be made in the form of «How are texts related in the context of the word «mutation» through its manifestations?» Texts with numbers in the left column of the sub context on Fig. 2 a) are linked in the context of the word «mutation» by means of the words indicated in grey rectangles in the concept lattice.
The lattice in Fig. 2 (b) is trivial. It has only one layer and all concepts are independent. The three-element AMR scheme does not reveal the connections of texts in sufficient detail. For comparison, the same request was processed on a formal context built for the five-element AMR scheme (3).
Fig. 3 shows the fragment of the association built on the «mutation» cluster for the fifth element of the five-element AMR scheme. The numbers of occurrences of certain words in context points are shown in the summary table in Fig. 3 a). So the word «phenotype» occurs in 5 points and in some documents, including document No 51.
Processing a query in a six-dimensional context reveals a larger number of words that link texts. The corresponding two-dimensional formal sub-context has a larger size and its concept lattice shown in Fig. 3 b), is not trivial: it has a hierarchy of concepts. Fig. 3. A fragment of association built on the «mutation» cluster the five-element AMR scheme and its concept lattice Comparing the lattices in Fig.2 (b) and Fig. 3 (b), we see that, for example, in Fig. 2 (b) texts 231, 138, 51 are included in the same concept with the word «phenotype» combining them, and in the lattice in Fig. 3 these texts form three different concepts with a large number of unifying words.
At the same time, the concept that includes text 231 is more general for the concepts that include texts 51 and 138. The lattice in Fig. 3 b) can be named as "What is affected by mutation in different texts".
Based on these results, the following conclusions can be drawn. 1. The problem of finding dependencies between texts can be solved by clustering data of polyadic formal context using data associations. 2. The informativeness of a five-element AMR scheme is qualitatively higher than that of a three-element AMR scheme.
It is obvious that due to the universality of this text analysis tool, it can be used in various other tasks of relation extraction.
In this paper, we propose a method for constructing and applying polyadic formal contexts on natural language tests. The method uses conceptual graphs acquired from texts and, together with AMR-schemata, these graphs constitute a data source for polyadic formal contexts.
Polyadic formal contexts constructed by this way may be used as a tool for multimodal clustering. This tool was tested here on the problem of finding dependencies between texts.
It should be noted that the use of conceptual graphs makes it possible to construct AMR schemata of greater length than those considered in this paper. This will allow for the implementation of polyadic formal contexts that reflect the content of the modeled text more fully and, accordingly, to extract more complete information from it. The method can be applied in Question-answering systems, in which natural language queries correspond to the logic of AMR schemes.
Acknowledgement. The reported study was funded by Russian Foundation of Basic Research, the research project № 19-07-01178 and RFBR and Tula Region according to research project № 19-47-710007.