This paper examines and analyzes approaches for constructing network of terms as an ontological subject domain model. In particular, new approaches and rules for determining the syntax and semantic links between terms in the text and the directions of these links between nodes in undirected networks of terms constructed from terms of a thematic text corpus, are proposed and researched. Also, one of the methods for creating terminological ontologies - the algorithm for building the thematic networks of natural hierarchies of terms based on analysis of texts corpora - is considered and used to build a directed network of words and phrases (separate unigrams, bigrams and threegrams). The wellknown fairy tale “The story of Little Red Riding Hood” is provided as examples to demonstrate an accuracy of the proposed rules. The Python programming language and its separate functions of a specialized add-in - the module NLTK (Natural Language Toolkit open source library) is used to create the software realization of the proposed and considered approaches and methods. Using the software for modelling and visualization of graphs - Gephi, the built directed networks of terms were visualized for better visual perception. The proposed approach can be used for automatically creating terminological ontologies of subject domains with the participation of experts. Also, the research result can be used to create personal search interfaces for users of information retrieval systems and also can be used in navigation systems in databases. It should help users of such systems simplify the process of searching the relevant information.
methodologies and techniques of computerized text processing and analysis. The modern software is increasingly in need of ready-made solutions to improve its systems.
It should be noted that it is very important to formalize the knowledge of some
subject domain while its studying. This process of representing, formal naming and
definition of the categories, properties and relations between the concepts, data and entities
is known as ontology modeling of the subject domain. A network of terms can be
considered as a model of some subject domain. In this network of terms, nodes correspond
to the individual words and phrases in the text and the edges to the links between them.
The process of ontology creating is usually very complex and resource-intensive, and
besides this, it is still an unsolved scientific and practical problem [
Due to the complexity of natural language, the determination of the syntax and semantic links between nodes that correspond to the terms in the text and the determination of the directions of these links is also an equally complex and open problem of conceptualization.
The purpose of this work is to propose and present new approaches for determining the directions of links between nodes in undirected networks of terms built from words and phrases (separate unigrams, bigrams and trigrams) of a thematic text corpus. 2
There are several approaches for transforming the texts into a network of terms and
different ways to interpret nodes and connections [
In this work, the compactified horizontal visibility graph (CHVG) algorithm for creating terminological ontologies of subject domains for key terms (separate unigrams, bigrams and trigrams) is used. 2.1
The horizontal visibility graph (HVG) algorithm [
In the work [
Two nodes ti and tj corresponding to the elements of the time series xi and xj, are is connected in a HVG if and only if, when xk < min(xi; xj) for all tk (ti < tk < tj).
In the third stage, the network that obtained on in the previous steps is compactified: the nodes that correspond to the same terms are combined into a single node. The obtained undirected network of terms is called the compactified horizontal visibility graph (CHVG) (see fig. 1).
Рис. 1. Stages of a building of the compactified horizontal visibility graph [
Thus, the CHVG algorithm allows building an undirected network of terms in case, when the numerical values are assigned to separated words or phrases (separate unigrams, bigrams and trigrams) of a thematic text corpus. 2.2
Languages we speak and write are made up of several words often derived from one another.
When a language contains words that are derived from another word as their use in the speech changes is called Inflected Language. It is clear to understand that an inflected word(s) will have a common root form.
In this section, we briefly describe the main parts of processing text documents such as tokenization, part-of-speech tagging, lemmatization, stop words removal, stemming process and terms weighting.
For preliminary lexical analysis, breaking text up into its single words (tokens) – tokenization, is made.
Lemmatization usually refers to doing things properly with the use of vocabulary and morphological analysis of words, normally aiming to remove inflectional endings only and to return the base or dictionary form of a word, which is known as the lemma. A lemma (plural lemmas or lemmata) is the canonical form, dictionary form, or citation form of a set of words.
For example, "runs", "running", "ran" are all forms of the word "run", therefore "run" is the lemma of all these words. Because lemmatization returns an actual word of the language, it is used where it is necessary to get valid words.
In this work, “WordNet Lemmatizer” provided by Python NLTK was used to lemmatize the tokens. “WordNet Lemmatizer” uses the WordNet Database to lookup lemmas of words.
Tokenization and lemmatization are usually the initial stages of word processing
because they allow you to work with a word as a single entity, knowing its context [
POS tagging is one of the first steps in computer text analysis.
Before lemmatization, it is necessary to provide the context in which you want to
lemmatize that is the parts-of-speech (POS) [
In corpus linguistics, part-of-speech tagging (POS tagging or PoS tagging or POST), also called grammatical tagging or word-category disambiguation, is the process of marking up a word in a text (corpus) as corresponding to a particular part of speech, based on both its definition and its context—i.e., its relationship with adjacent and related words in a phrase, sentence, or paragraph. A simplified form of this is commonly taught to school-age children, in the identification of words as nouns, verbs, adjectives, adverbs, etc.
In general, PoS tagging algorithms are divided into two distinct groups: rule-based
and stochastic. E. Brill's tagging method [
“Part of Speech tagging” is one of the more powerful aspects of the NLTK module in the Python programming language. Basically, the goal of a POS tagger is to assign linguistic (mostly grammatical) information to sub-sentential units - tokens.
Also, after the stage of pre-procession of the textual documents and the extraction of key terms in this study, it is proposed to remove stop words that have no semantic strength, that is, informationally unimportant ones, as well as bigrams containing at least one stop word and trigrams that start or end with a stop word. In general, stop words are words that do not contain important significance to be used in Search Queries. Usually, these words are filtered out from search queries because they return a vast amount of unnecessary information. Mostly they are words that are commonly used in the English language such as 'as, the, be, are' etc.
The stop dictionary used in this work was based on different stop dictionaries, which are available at: https://code.google.com/archive/p/stop-words/downloads/; http://www.textfixer.com/tutorials/common-english-words.php.
It should be noted, that each programming language will give its list of stop words to use. In this work, the “SnowballStemmer” (stemmer that is realized in Python in NLTK librаry – Natural Language Toolkit librаry) was also used to ignore stop words.
Also, the formed stop dictionary was expanded by adding other stop words that were identified by experts within the considered subject domain.
After the stages described above, for combining the words that have a common root
into a single word it is proposed to carry out the process of stemming. Stemming is the
process of reducing inflection in words to their root forms such as mapping a group of
words to the same stem even if the stem itself is not a valid word in the Language [
The goal of both stemming and lemmatization is to reduce inflectional forms and sometimes derivationally related forms of a word to a common base form.
However, the two processes differ in that stemming most commonly collapses derivationally related words, whereas lemmatization commonly only collapses the different inflectional forms of a lemma.
If confronted with the token "saw", stemming might return just s, whereas lemmatization would attempt to return either "see" or "saw" depending on whether the use of the token was as a verb or a noun.
To avoid the confusion described above, in this work the lemmatization process precedes the stemming process.
Several stemming algorithms can be distinguished in terms of performance,
accuracy, and how stemming problems are overcome [
The most common algorithm for stemming English, and one that has repeatedly been
shown to be empirically very effective, is Porter's algorithm [
The pre-processing stages described above allows normalizing the text corpus. 2.3
After the pre-processing stages, the weighting and extraction of the key terms are made.
To form a time series, the function that reflects the term to number, this study uses the
modification of classic statistical weight indicator TF-IDF (from English, TF is Term
Frequency, IDF is Inverse Document Frequency) [
This approach allows having a high statistical indicator of importance for informationally-important in global context elements of the text.
As was mentioned above, the determination of the directions of links is a complex and open problem of ontology creation. Below, we consider several new approaches for determining the directions of links between nodes in undirected networks of terms built from words and phrases (separate unigrams, bigrams and trigrams) of a thematic text corpus.
Let G be the undirected network of terms that built according to the described above rules: G:= (V, T) where V is the set of nodes, T is the set of the unordered pairs of nodes from the set V that correspond to the causal links between the nodes.
It is supposed that a causal link exists in the direction from the node ti to the node tj for ∀ , : (ti, tj)∈ T if:
1. the numerical value of the node ti that corresponds to: a) degree [
2. within the sentence, the term to which the node ti corresponds precedes the term to which the node tj corresponds;
3. the term to which the node ti corresponds is shorter than the term to which the node tj corresponds.
One of the methods for creating terminological ontologies – the algorithm for
building the thematic networks of natural hierarchies of terms based on analysis of texts
corpora – is used to build the directed network of words and phrases (separate unigrams,
bigrams and trigrams) according to the third rule. The work [
The proposed approaches for determining the directions of links in undirected networks of terms was tested on the example of the English-language text, namely – the wellknown fairy tale “The story of Little Red Riding Hood”.
According to the described above method, the text pre-processing and the extraction of the key terms (separate unigrams, bigrams and trigrams) were made (Table 1, 2 and 3).
The following results were obtained after building the directed network according to the first rule for different measures of network nodes (for the degree – fig. 2; for the HITS – fig. 3; for the PageRank – fig. 4). Using the software for modeling and visualization of graphs – Gephi (https://gephi.org), the built directed networks of terms were visualized for better visual perception.
Fig. 2. Fragment of the directed network, which built according to the first rule for node degree.
After analyzing the obtained results, it was found that the directed network, which built according to the second rule more precisely reflects the directions of links that exist between the terms in the considered text, than the network, which built according to the first rule. The network of natural hierarchies of terms has its peculiarities and advantages, so it is difficult to compare it with the networks built according to the first two rules. Taking account into the naturalness of links that determined in such a network, we can talk about their syntactic adequacy.
Considering, for example, the directions of links determined for key terms, we can see that according to the first rule, the links between “wolf”-“grandmother”-“red” are as follows (see fig. 2,3,4): for the degree, HITS and PageRank – the “grandmother” influences on the “wolf” and the “red”, and the “red” influences on “wolf”. It does not correspond to the real directions of links that exist in the text in terms of content analysis. While, according to the second rule, the “wolf” influences on the “grandmother” and the “grandmother” influences on the “red”, which corresponds to the content of the considered text.
In comparison with other rules, the rule for determining the directions of links in undirected networks of terms, when within the sentence, the term to which the node ti corresponds precedes the term to which the node t j corresponds (where t j ( ti , t j ) ∈ T) is more informative among the first two rules. It is because the links determined according to this rule more precisely corresponds to the content of the considered text according to experts.
After studying the proposed rules for determining the directions of links in undirected networks of terms, it was found that the second rule more precisely reflects the directions of links, which correspond to the content of the considered text according to experts and is more informative. On the example of the English-language text – the wellknown fairy tale “The story of Little Red Riding Hood” the undirected network of terms was built. Using the proposed rules for determining the directions of links, the directed networks of terms were obtained from undirected networks of terms. Informative content of network links built according to the second proposed rule is higher among the other two rules according to experts. Taking account into the naturalness of links that determined in the network of natural hierarchies of terms, we can talk about their syntactic adequacy.
The directed networks of words and phrases built according to the proposed approach can be used for automatically creating terminological ontologies of subject domains with the participation of experts. Also, the research result can be used to create personal search interfaces for users of information retrieval systems and also can be used in navigation systems in databases. It should help users of such systems simplify the process of searching the relevant information.
As the task of improving the accuracy of determining the directions of links between nodes in undirected networks of words and phrases is actual, then it is planned to continue working in this direction, developing new and modifying existing approaches.