Building a lattice-based taxonomy over a text corpus with formal concept analysis (FCA) methods requires preliminary text processing that would enable construction of a context. We consider several natural language processing methods aimed at automatic attribute acquisition from texts. In particular, we derive attributes of three types: frequent words, latent topics and named entities. Afterwards, we construct a context for each type taking documents in the corpus as a set of objects. Then the corresponding concept lattices are built and pruned with the help of stability index in order to improve the readability of the diagrams. The proposed technique is illustrated on a collection of 26 texts in English dealing with political domain. In this case, the technique serves as a tool for deeper understanding of the interests of di erent political actors producing political texts by clarifying the connections between notions they use in them.
Constructing a taxonomy over a text corpus requires preliminary text
processing. In this paper we explore capabilities of several natural language processing
methods for extracting a set of attributes from the documents each taken as
an object. The rst method is keyword extraction which is based on the
Vector Space Model [
Here, we apply these three methods to a collection of 26 texts dealing with political domain. After constructing an object-attribute matrix and building a lattice, tering based on the stability index is applied in order to improve the readability of the diagrams. ?? This work is supported by project 10-04-0017 of the Scienti c Foundation of the State University-Higher School of Economics \Discrete mathematical models for political analysis of democratic institutions and human rights".
Our primary data consists of 26 texts corresponding to speeches of European and American leaders and spokepersons during the 2007-2010 period addressing their relations with Russia. These documents were collected by experts in political science as part of an interdisciplinary research project. From the viewpoint of political studies, the aims were (1) to de ne the context in which Russia is addressed in the speeches of Western leaders and international organizations, (2) to analyze the role and importance of democracy and human rights agenda. However, this text corpus is used here primarily as the testing ground for di erent methods of text analysis, and will be expanded in further research in order to reach a higher level of validity for the conclusions. 2
In The Vector Space Model (VSM) [
; nij where nij is the number of occurrences of the word j in document i; the denominator is the total number of words in the document i.
Constructing the Context and Stability-based Pruning Formal
Concept Analysis (FCA) methods have been proven useful for representing a
meaningful structure of a given knowledge community (such as a scienti c community
[
Using the database of 26 political texts we have built a context made of documents and terms mentioned in each document most frequently. In particular, we took 20 most popular terms for each document according to tf measure. The resulting context contains 26 documents and 249 terms which yields a lattice of 453 formal concepts.
The number of concepts is too large to be shown in a diagram. In order to
obtain intelligible diagrams, we apply the pruning technique based on the notion
of concept stability [
A j C0 = Bgj 2jAj
The basic stability-based pruning method is to remove all concepts with stability below a xed threshold. Of course, stable concepts (i.e., satisfying the chosen stability threshold) do not always form a lattice. However, this fact does not in uence interpretation of results.
The reduced substructure featuring the 31 most stable concepts is presented
in Fig. 1. The diagrams were produced with Concept Explorer.[
In this section we address an issue of probabilistic modeling of text. Its basic
idea is that documents are represented as random mixtures over latent topics,
where each topic is characterized by a distribution over words. Such probabilistic
topic approach to document modeling was introduced as probabilistic Latent
Semantic Analysis (pLSA) [
with
Matlab
Topic
Modeling
Toolbox: right nation georgia russia global secur europ human unit russian russian polici today european govern nuclear intern interest institut member union peopl america georgian energy issu challeng idea democraci american territori medvedev e ort afganistan point work interest south issu respons strateg thing women futur order rule achiev face bring democrat weapon process trust approach matter global
After applying the LDA model to the text corpora we construct a context taking topics as attributes for the documents|a topic is assigned to a document if the total number of times that words in this document were assigned to a particular topic exceeds a xed threshold. The resulting substructure of the corresponding pruned lattice is presented in Fig. 2. Lists of words in squares represent the rst six words assigned to a topic with the highest probability.
According to this lattice the most actual topics are those connected with European Union (topic represented by terms \europ", \european", \union", \idea" and assigned to 12 documents), global problems (\global", \polici", \institut", \issu", \e ort", 14 documents) and security issues (\secur", \today", \member", \challeng", \afganistan", \starteg", 14 documents), as well as energy resources (\russia", \relat", \russian", \interest", \energi", \medvedev", 12 documents). There is a concept corresponding to the topic of Russian-Georgian con ict which contains 5 documents. In addition there is an isolated concept related to economic development of China (\peopl", \work", \commit", \econom", \help", \china", 9 documents). 4
Name Entity Recognition (NER) is the process of nding mentions of xed types of information in human language. From the 26 texts, we extracted 38 paragraphs that touch issues related to Russia. The paragraphs were processed with GATE2 system and three types of named entities were extracted|names of persons, organizations and geographical objects. We construct a context taking paragraphs as objects and organizations, persons and locations mentioned in a certain paragraph more than a xed number of times as attributes. The resulting pruned lattice is presented in Fig. 3.
From this lattice it can be noticed that Europe and European Union are the most discussed topics as it appeared in previous results. It is worth noticing that 2 GATE: General Architecture for Text Engineering: http://www.gate.ac.uk/ the United Nations (UN) is mentioned only in the context of Afghanistan, which in its turn is mentioned solely in the context of NATO. Topics corresponding to China and Ukraine form isolated concepts. 5
The rst method based on frequent words allowed us to identify what questions are raised most frequently by European and American leaders while talking about Russia, whereas latent topic modeling allowed us to specify and describe these issues more thoroughly. The results obtained with named-entity lattice are rather similar. We concluded that it could be more useful to merge NER with the LDA model, for instance, by taking named entities as a set of objects and topics as a set of attributes. Expanding the corpus of the texts and combining NER with the LDA model can be useful in testing the hypotheses addressed in political studies.
On the whole, building lattice-based taxonomies using various language processing techniques is promissing for obtaining additional knowledge regarding open and hidden intentions of political actors. What is important for social sciences is that the knowledge is obtained without collecting any additional information and by a \neutral" tool|through deriving deeper connections between notions used by the actors in their texts.