The paper presents the intermediate results of the research, the final goal of which is to develop the universal algorithm for process diagrams automatic visualization by text description of these processes. The purpose of this study is to check the use of verbs as markers for the semantic labeling of long fragments in scientific texts.
An effective system of collecting, storage and data processing of scientific observations will raise any natural-science research to essentially new level. The description of objects and procedures is presented in the form of natural language texts. Therefore automatic transformation of texts into more effective representations (such as activity diagrams, state diagrams, use case diagrams, IDEF0 diagrams, etc.) is required to reduce the cost of specialized information systems creation.
Current researches in the field of process descriptions extraction from
natural language texts are oriented to work with the news bulletins [
To reach these goals, it is necessary to allocate text fragments with various assignments: – static (descriptions of objects, definitions) for ontology extraction; – dynamic (description of processes, techniques and research procedures) for activity diagrams and other process diagrams extraction.
A text can be divided into various fragments of these types with the use of
clustering on the base of statistical analysis of parts of speech distribution[
The purpose of the research is to check the possibility of using verbs as markers for different types of fragments.
For the illustrations we used Bykov N. I., Popov E. S. Observing the dynamics of snow cover in protected areas of the Altai-Sayan Ecoregion. Methodological guidance. Krasnoyarsk. 2011. 64 pages. The text consists of 1257 sentences. Parser identified 196 different verbs. 2
Let V be the set of all natural language verbs. Scientific text T is represented as an ordered set of natural language sentences T = hski, where sk is kth sentence in the text. Let Vk ⊂ V be the set of verbs in the sentence sk. For each verb let’s define the list Ev = hsk|v ∈ Vki. This is an ordered list of sentences that contain a verb v. |Ev| is the number of occurrences of the verb v in the text T . Since the object of study is the verb distribution in the text, the cases of multiple use of a single verb within a sentence can not be ignored. Text neighborhood Tv = hsii of the verb v is an ordered set of sentences si, such that ∀si ∃sk ∈ Ev and |k − i| <= , is a non-negative integer. All the verbs from the text T are divided into three groups. The first group contains rare verbs Vunic. The number of occurrences |Ev| in the text for these verbs is below the border β: |Ev| < β. The second group includes common verbs Vcommon. These verbs get the largest values of |Ev|, and are distributed relatively evenly within the text. Typically, these are parts of collocations from scientific speech style, such as ”ОСУЩЕСТВЛЯТЬ” (”TO CARRY OUT”), ”ПРОИЗВОДИТЬ” (”TO MAKE”). The third group contains verbs-markers Vmarker. Those verbs-markers are present in the text in sufficient quantities and are unevenly distributed. These verbs can also be parts of collocations from scientific speech style.
Let each sentence sk of the text T be assigned to some cluster c from a finite
set of clusters C. For example, the set of clusters can be obtained by clustering
on the base of the distribution of parts of speech along the text (described in
detail in [
The verb vm is marker of cluster c, if |cvm |/|Tvm | > σ and ∀a ∈ C|avm |/|Tvm | ≤ σ. The values of and σ are parameters of marker detection algorithm and depend on the method of obtaining clusters C.
The text nest of verb-marker vm is the set of verbs: Nvμm = {v|Ev ∩ Tvμm 6= ∅}. 3
The mathematical model described is realized in algorithms for automatic labeling of text fragments and for construction of text nests of verbs. In the software complex the sentence sk is implemented as a parse tree of Dialing parser1. The table 1 presents the lists of verbs for the three clusters. The window size for clustering L = 120 sentences. The algorithm parameters values is = 7 sentences and σ = 0.9.
Let’s consider the example of a nest for marker ” ВЫЧИСЛЯТЬ” (” TO CALCULATE”) for μ = 7 sentences: 1 http://aot.ru/ НАПОМНИТЬ, ОТСУТСТВОВАТЬ, РАССЧИТЫВАТЬСЯ, ОКРУГЛЯТЬ, ЗАПАСТИ, ПРЕДСТАВЛЯТЬ, УЧИТЫВАТЬСЯ, ОКАЗАТЬСЯ, ПРОБИВАТЬСЯ, ПОЗВОЛЯТЬ, ВЫБИРАТЬ, ПОДСЧИТЫВАТЬ (TO REMIND, TO BE ABSENT, TO BE CALCULATED, TO ROUND, TO STORE, TO PRESENT, TO BE CONSIDERED, TO APPEAR, TO BREAK THROUGH, TO ALLOW, TO CHOOSE, TO COUNT) The nest obtained shows that the set of verbs, which is located around the marker, belongs to the calculations and laboratory processing of research results for snow cover observations.
The algorithm of fragments labeling and nests construction has been checked using the test set containing 15 scientific texts of various authors and subjects. Selected verbs-markers are consistent with the expert annotation of the fragments content. Verbs-markers can be used for semantic labeling of automatically separated fragments, although some of them have no semantic value and are just stylistic features of a specific text. In the future we are planning to use verbsmarkers to improve the accuracy of fragments boundaries determining. The nests of verbs received on the basis of the model presented will be used in algorithms of processes visualization using their text descriptions. Аннотация В статье представлены промежуточные результаты исследования, конечной целью которого является разработка универсального алгоритма для автоматической визуализации диаграмм процессов по текстовым описанием этих процессов. Цель данного исследования проверка возможности использования глаголов в качестве маркеров для семантической маркировки длинных фрагментов в научных текстах. Ключевые слова: автоматическое фрагментирование текста, text mining, глаголы-маркеры, динамические параметры текста.