The problems of extracting the most complete information from the semantic library by accounting for related documents are considered. Expert knowledge encrypted in the subject area can be made available when the user obtains additional information from linked documents. A feature of the approach is the use of a shallow neural network algorithm to expand the search query in mathematical subject areas, where expert knowledge is available with a significant scientific background of users. The solution to this problem can be achieved by means of semantic analysis in the knowledge space using machine learning algorithms. The paper investigates the construction of a vector representation of documents based on paragraphs in relation to the data array of the digital semantic library LibMeta. Each piece of text is labelled. Both the whole document and its separate parts can be marked. Since the problem of enriching user queries with synonyms was solved, when building a search model in conjunction with word2vec algorithms, an approach of “indexing first, then training” was used to cover more information and give more accurate results.
The history of research the problems of expanding the request for the most complete
coverage of information is quite long [
Many approaches have been developed with the advent of artificial intelligence
algorithms and corresponding programming tools in this area [
The next stage of introducing artificial intelligence into knowledge systems is due
to the use of neural network algorithms [
More complex links and structures are embedded in scientific libraries, which is
dictated by the logic of subject areas and requires more careful processing of links to
provide users with advanced query capabilities [
This work is devoted to the use of shallow neural network algorithms [
The LibMeta resources include a thesaurus on ordinary differential equations
(ODE), dictionaries for special functions of equations of mathematical physics. All
dictionaries are semantically linked to a mathematical encyclopedia [
This paper presents a search model (part 2), outlines a technique based on the use
of algorithms for vector representation of texts [
2
The construction of the search model in LibMeta is based on three main key points, namely: converting documents to searchable format; requests are presented in a format that allows expressing the user's information needs; the assessment of the compliance of the document with the request. In our case, for the preparation of documents, preprocessing of full texts was carried out to remove the publisher's markup and highlight the main parts of the text. Then a full-text document index was created, which allows you to efficiently load and store data and provide quick access to it. Queries written in natural language are used, which can be enriched with synonyms by the system. The assessment of the compliance of a document with a request is subjective and depends on the method used.
One of the most commonly used document and query presentation models is the
vector space model [
In vector notation, each word is associated with a weight, which can be calculated
in different ways. One of the most commonly used algorithms is the TF-IDF [
The main problem of any search model is to provide relevant results in relation to
the user's information needs: from query analysis to ranking search results. This work
is devoted to options for resolving this problem. One of the modern approaches is to
use neural networks for text processing, since text is an example of data that can be
parsed into smaller structures such as paragraphs, sentences, words, etc. depending on
the text. This approach to text processing allows you to capture the semantics of the
text, since closely related words or fragments of text occur in the same context and lie
side by side in vector space. The search model used in this work is based on the
vector representation of words and documents built using the word2vec [
Integration of neural network and index can be done in the following ways: first training on the corpus of texts, then indexing the texts and share them in the search; indexing first, then training on indexed data and sharing in search; first training, then extraction / creation of useful resources by the trained network, and then indexing of all resources, both new and original.
Since we were solving the problem of enriching user queries with synonyms, in the LibMeta system we used the “indexing first, then training” approach to provide more results and more accurate results, based on extended queries on the one hand. On the other hand, using the extended version of word2vec in conjunction with the LibMeta search engine, it becomes possible to give users smarter recommendations based on the documents found. This approach to sharing the index and search engine and neural network allows for relevant models and ranking functions that adapt well to the underlying data. The version of the model built on the LibMeta search index using word2vec algorithms, hereinafter we will be abbreviated as wsgMath.
Figure 1 schematically illustrates the operation of a search based on a neural network, which receives a query string as input, then returns synonyms to the query using the model built by word2vec. In another case, a document on a vector representation can be submitted to the input, which, using the constructed model, gives recommendations in the form of a list of documents similar to it.
3
Studies show [
For convenience, we will use the term “paragraph” to denote a paragraph, as such, but also for fragments of a paragraph or several phrases from the text. As applied to our field and the specifics of the structure of a mathematical text, these can also be theorems, lemmas, etc.
Note that the term “important” fragment will also be used. In scientific texts, this is an abstract, introduction, conclusion, theorem, etc. This term is defined since the specified elements of a scientific publication will be used as defining for documents belonging to a certain subject area.
Content for research is the resources of the LibMeta digital library [
Therefore, to construct wsgMath, taking into account the context for paragraphs,
we used a version of the word2vec algorithm which is a generalization (extension) of
the original doc2vec algorithm [
The procedure for highlighting such content will be used to offer the user semantically related documents. It is essential that without the application of the algorithm for highlighting related content, such documents will not be displayed in the search results by request, since they may not contain keywords from the query or not directly related to a certain subject area in other terms.
The peculiarity of common search models, such as the vector space model with TF-IDF, is that they only take into account individual terms. This approach does not always lead to optimal results because contextual information is discarded. The word context is understood as N words in the text before the word for which the vector is constructed, and N words after this word. In contrast to the TF-IDF model, the individual elements of the vector are not interpretable, but the distance between the vectors is investigated, which is interpreted as the semantic proximity of words.
Based on the vector representation, the proximity of the texts is measured. Using the search index and vector document representation together leverages the ability of these views to capture the semantics of text when building search models that are well adapted to the data.
The main metrics for measuring the proximity of texts are cosine distance and Euclidean distance, which are used to capture semantically similar words, sentences, paragraphs, etc.
4
The analysis of mathematical texts is conventionally considered as the analysis of the actual mathematical text as a whole, the analysis of formulas as a “separate language” for the representation of mathematical knowledge and the establishment of semantic links between the text and formulas. Further, only the analysis of the mathematical text as a whole is considered.
To extract synonyms for query terms from the constructed model, lexical and grammatical templates were used, which are one of the recognized methods for extracting links from text [34-38]. Based on the idea of using such patterns, we investigated the task of extracting synonyms of concepts and extracting / constructing simple patterns from them to identify relationships.
The implementation of the model consists in the application of an iterative research algorithm, which will be called iraWsgMath below. We list its main stages:
allocation of synonyms of terms As an example, we will demonstrate the query “Cauchy problem” (For the convenience of the reader, the examples have been translated into English, but the work was done for texts in Russian. In Russian the considering term is “задача Коши”), which consists of two words, “problem” and “Cauchy”, each of which has its own synonyms, which are presented in Table 1.
The third column presents the query context as one unit Cauchy problem (задача Коши). Extracting its synonyms, it is clear that the list consists of words where the adjective boundary falls, which also occurs in the synonyms of individual words in the first two columns.
In this case, the term “Cauchy problem” itself has the following synonyms: “Cauchy equation”, “Cauchy inequality”, which were determined on the basis of high estimates of the proximity of the following pairs of synonyms, for example, for a pair (problem, equation), the proximity estimate is 0.84.
Note that when constructing synonymous terms, synonyms of the word Cauchy were not used, since it was defined as a named entity Cauchy based on a dictionary that includes a list of persons mentioned in the mathematical encyclopedia. But at the same time, we note that Riemann got into the synonyms of Cauchy.
determination of classes of synonyms by parts of speech
Lexical and grammatical templates were used to extract synonyms for query terms from the constructed model. They are one of the recognized methods for extracting links from text [34, 35]. Based on the idea of using such patterns, we investigated the task of extracting synonyms of concepts and extracting / constructing simple patterns from them to identify relationships. Consider a link extraction pattern based on a simple adjective <term> pattern that most often indicates generic links. The original term is a generic concept, and the combination corresponding to the pattern is a specific concept [36, 37, 38].
Each word was considered separately, the synonyms are filtered by parts of speech and a possible synonym (candidate) of the term is formed from them. After that, a sentence was formed for the term and its synonyms based on the selected templates.
Based on these synonyms, the following sentences were formed, which were obtained in accordance with the pattern “adjective <request term> <synonyms of the request term>”: [Cauchy boundary value problem, Cauchy boundary equation, Cauchy boundary inequality] (In Russian was used word “краевой”).
When compiling an extended query, it was also proposed to use the adjective boundary as a synonym, therefore additional queries were used: [Cauchy boundary value problem, Cauchy boundary equation, Cauchy boundary inequality]. (In Russian was used word “краевой”).
selection of patterns of “capture” of links
The <term> verb pattern was considered to analyze and construct more complex relationship patterns in the <term> verb <term> thesaurus. Using this pattern to fill the links with it requires a separate analysis and is beyond the scope of the word2vecbased algorithm considered in this article.
In the process of training the model, several verbs were defined to identify patterns using the considered algorithm. When analyzing context-sensitive synonyms, the list of emerging verbs was rather limited, which is not surprising due to the specifics of the subject area. The list of these verbs is limited to such as: apply, use, apply, base, prove, consider, consider, define, depend, be, embody. Also often used are verbal nouns formed from the listed verbs: application, use, application, basis, definition.
improving the “quality of terms” and checking them
To improve the search for extended domain terms matching templates for verbose terms in the thesaurus, possible spellings of terms were considered, for example, for “ordinary differential equation”, the possible options are “ODE”, “ordinary DE”, etc. All possible spellings were explored as separate terms. Since there are few such terms in the studied subset of terms, they did not have a significant meaning on the results.
Validation of the model and the links it retrieves was performed based on the thesaurus ODE.
The problem of synonyms and their extraction using word2vec with search index is
covered in more detail in [
5
The combined use of the full-text index [40, 41] and the search model wsgMath makes it possible to extend the original query with synonyms. Extending queries with synonyms without wsgMath requires pre-compiled synonym dictionaries. You can use resources such as WordNet (https://wordnet.princeton.edu/) or RuWordNet (https://ruwordnet.ru/ru), but the main problem is that synonyms from pre-compiled dictionaries are not tied to the data being indexed and their use does not improve the results.
Figure 2 shows the main steps of forming a model wsgMath for generating query synonyms in LibMeta content. The query string coming from the full-text search interface goes through the Analyzer. Analyzer is a functional part of the model, where the basic operations for interacting with the wsgMath model are performed. All operations described in points in the previous section refer to its main functionality.
The Analyzer splits a string into words, analyzes and transforms them. From the wsgMath model, synonyms for words are extracted and filtered, an extended query is formed, with the help of which the corresponding documents are extracted from the full-text index.
A user's information need is defined as a chain of requests that leads him to the information he needs. Each subsequent request in this chain is a refinement of the previous one.
A real information request, as a rule, consists of an initial request and clarifications. Let's consider an example, when the primary query leads to excessive information noise, and the refinement allows you to get a more pertinent answer, and compare the search results using the wsgMath model and without it. For comparison, statistical characteristics are calculated (denote score) obtained using the TF-IDF algorithm.
The example below demonstrates three lists (List 1-3) with different scores depending on how the query is expanded. For example, when searching for the test query “Cauchy problem”, the user enters the qualifying query “Cauchy boundary value problem” and finds the information of interest. Based on the fact that the search index contains 3654 scientific articles, of which only 637 contain a mention of the “Cauchy problem”. Of these, 59 pieces were selected for the user, since the query words were found in significant parts of the document (title and annotation). With this approach, the document of interest to the user is in 18th place. Part of the list is shown below, “score” value shows how well a document matches the request and is calculated based on statistical characteristics such as TF-IDF. 1. The Cauchy problem for the system of equations of the theory of elasticity and thermoelasticity in space score = 0.65376675 2. The Cauchy problem for the system of thermoelasticity equations in space score = 0.64415324 ………………………………………………………. 18. On the Well-Posedness of a Boundary Value Problem on the Line for Three
With the refinement query “Cauchy boundary value problem”, the list of results looks as shown below, and the document of interest is moved to the fifth position, while the number of documents satisfying the query text is reduced to 338, while the user is recommended only 20 of them.
List 2: 1. Projection procedures for non-local improvement of linearly controlled processes score = 0.8902895 2. On one method of constructing parametric synthesis for a linear-quadratic optimal control problem score = 0.8708762 …………………………………………
Let us consider the situation when the query “Cauchy problem” is extended by synonyms and is transformed into the form “boundary”, “problem or equation or inequality Cauchy” (in Russian: “краевая или граничная”, “задача или уравнение или неравенство Коши”) using the wsgMath model. In parentheses in an extended query, synonyms are listed, connected by a logical operation OR. The presence of at least one of these synonyms is required. This approach insignificantly increases the completeness of the answer, and the accuracy also increases, therefore, the degree of satisfaction of the user's need increases. The list of results obtained is displayed below and the searched document is in second place. The number of documents corresponding to the request is 395 and the user will receive the desired answer already in the first positions, while the size of the issue by the system is 65.
1. On a positive radially symmetric solution of the Dirichlet problem for one nonlinear equation and a numerical method for obtaining it score = 0.9809638
score = 0.9587569 3. On a positive radially symmetric solution of the Dirichlet problem for one nonlinear equation and a numerical method for obtaining it score = 0.9512307
This example illustrates the effect of this approach already at the level of extending queries with synonyms based on indexed documents. With this approach, all suggested synonyms are found in the search engine index and the query extension is guaranteed to offer answers to the user's queries.
The use of the extended version of word2vec (doc2vec or paragraph2vec, in
different sources in different ways) [
Let's take a closer look at the process of ranking documents based on the wsgMath model when searching for similar documents. When a document enters the system, its current vector representation is retrieved, a search is performed, and the labels of the nearest documents are returned, the cosine distance of which exceeds a certain threshold, determined experimentally as 0.6. Below is the result of the work on the example of the document, which in the previous example was the desired one. As the closest to it, 9 documents were found whose cosine distance exceeded 0.6.
List 4: 1. Some classes of singular integral equations solvable in closed form cosineSimilarity = 0.8136491179466248 2. Riemann's boundary value problem for a half-plane with a coefficient exponentially decreasing at infinity cosineSimilarity = 0.8028532266616821 3. Algorithm for constructing a quasiregular asymptotic representation of the solution of singularly perturbed linear multipoint boundary value problems with fast and slow variables cosineSimilarity = 0.7246567010879517 4. Solution in closed form of an integral equation of convolution type in the hyperelliptic case cosineSimilarity = 0.6468908786773682 5. On biorthogonal systems generated by some involutive operators cosineSimilarity = 0.6454607248306274 6. On linear periodic systems in the plane having matrices of the required form cosineSimilarity = 0.6165973544120789 7. On integral equations for the Riemann function cosineSimilarity = 0.6134763956069946 8. Gakhov's equation for an exterior mixed inverse boundary value problem with respect to a parameter ... cosineSimilarity = 0.6059825420379639 9. On a nonlinear integral equation of the first kind cosineSimilarity = 0.6017340421676636
In Fig. 4 adds steps that include attribute search and how it interacts with the previously described search components. Attribute-based search delineates the boundaries in which documents are searched (by author, by year, etc.), then a transition to fulltext search can be performed on them, and/or its results can also be refined based on the similarity of documents.
6
Vector representation of documents is proposed to expand the search query, increase the coverage of information on demand. It is shown that the quality of an answer to a request is improved by taking into account semantically close text fragments.
The model proposed in the work was tested on primary data, namely, arrays of articles not systematized by subject matter. Note that the technology of processing and thematic classification of primary data using machine learning methods has been tested. This technology can be used for the subject classification of the texts of scientific articles in Russian and the comparison of selected subjects with the English-language classification by comparing the MSC and UDC classifiers.
Integration of neural network and search indexes makes possible to give users smarter results based on the identified relations among documents.
Also, the considered search model can be used for thematic processing, both primary texts of scientific articles, and already systematized, provided with keywords and links to classifiers. In the second case, this can help to identify interdisciplinary research, as well as erroneous assignments of the subject area, since not only secondary documents, but also the texts of articles (primary documents) are taken as the basis for thematic analysis.
Aknowledgement. The work is presented in the framework of the implementation of the theme of the state assignment “Mathematical methods of data analysis and forecasting” FRC CSC of RAS and partially supported by grant #20-07-00324 of the Russian Foundation of Basic Research. 34. Bullinaria, J.A., Levy, J.P.: Extracting Semantic Representations from Word Cooccurrence Statistics: A Computational Study, Behavior Research Methods, vol. 39, pp. 510–526 (2007). 35. Klaussner, C., Zhekova, D.: Lexico-syntactic patterns for automatic ontology building, Proceedings of the Second Student Research Workshop associated with RANLP, 109–114 (2011). 36. Raza, M.A., Mokhtar, R., Ahmad, N., Pasha, M., and Pasha, U.: A Taxonomy and Survey of Semantic Approaches for Query Expansion, in IEEE Access, vol. 7, pp. 17823-17833, (2019) https://doi.org/10.1109/ACCESS.2019.2894679. 37. Wang, C., Cao, L., Zhou, B.: Medical synonym extraction with concept space models https://arxiv.org/abs/1506.00528. (2015), last accessed 2021/07/27. 38. Mitchell, J., and Lapata, M.: Vector-based Models of Semantic Composition (2008). 39. Polozov I.K., Volkova I.A.: Applying word2vec technology to shifter extraction task. International research journal 4-1 (94) (2020). 40. Makinen, V.: Compact suffix array — a space-efficient full-text index. Fundamenta Informaticae 56(1–2), 191–210 (2003). 41. Makinen, V., and Navarro, G.: Compressed full-text indexes ACM Computing Surveys 39, (1), 1–79 (2007) https://doi.org/10.1145/1216370.1216372.