A well-known direction to define similarities of texts is based on considering matching fragments of texts (also called plagiarism) allows for defining narrow aspects of text similarities. But, the fact, that texts have the same semantics and can use, for example, synonyms or a set of other words to express the same idea left in the other hand of the many working software systems, that are aimed to define plagiarism. The paper is devoted to the representation of conceptual basics explaining how to define whether different texts are equal. Such conceptual basics are grounded on the idea of extracting facts from texts and using ontologies and growing pyramidal networks to match whether texts are equal. Growing pyramidal network, ontology, semantic cooperation, text, natural language CMiGIN 2022: 2nd International Conference on Conflict Management in Global Information Networks, November 30, 2022, Kyiv, Ukraine ORCID: 0000-0002-9873-6010 (O. Chebanyuk); 0000-0002-7155-9202 (V. Velychko); 0000-0003-3223-1391 (O. Palahin); 0000-00034992-0564 (S. Gnatyuk)
The growing values of text information require principally new approaches for information processing. One of the important conditions is to implement new practical approaches allowing to process of big values of textual information. On the other hand, social networks (Figure 1) are the sources of a very big amount of information that can be duplicated in different tweets, posts, pages, and boards. EMAIL:
Palahin);
2022 Copyright for this paper by its authors.
Different users can be customers for defining simulates of textual information with different aims. For example, a researcher using social networks such as ResearchGate or ScienceDirect can define the best papers contain the best explanation of facts towards their interests. Tourist can find the best hotel and places to see according to their requires. The developer can optimize the procedure of documentation search. Many other practical tasks and practical approaches can be solved in this direction. The owner of the mailing list can define spam. Also, there are many other applications for this task.
Consider working systems, allowing to processing of textual information from social networks. IBM
Watson Services-based systems [
One of the tasks of such a system is to classify a user request according to topic. Using such a system
one can classify news, and spam or assume that text is touching on some specific topic (for example
information about disaster). The procedures below must be followed in order to train the classifier
before using the Natural Language Classifier service in your application: assemble the training set of
data; develop and train the classifier; Inquire about the trained classifier; assess the outcomes; and
update the data [
The diagram's steps are listed as follows: 1. The user inputs news information as text into the web interface and selects the Classify Text button to have it categorized. 2. The program (enterprise back end) receives the news text for processing. 3. To determine which category of news is the greatest match for the supplied text, the business application (web service) contacts the Watson Natural Language Classifier service. 4. The enterprise application receives the answer from the Natural Language Classifier service. 5. The web application front-end receives the answer from the business application. 6. The user may examine the response thanks to the web interface, which also controls the data information and carries out certain front-end processing. 7. The user feedback request is forwarded from the web application front end to the web application back end. 8. To persist the input, the back-end of the web application contacts the IBM Cloudant® noSQL DB service. 9. The database responds to the enterprise application with the results of the insert operation. 10. The web front-end receives the answer from the business application. 11. The user interface shows the user the findings.
Take into account other instances when such a classifier has been used while working with spam systems.
1. Using a web browser, the user navigates to the SPAM Classifier program [
The training data for the classifier [
Take language categorization as an example in an application that supports a Q and A procedure [
1. Through the online interface, a user asks a question about healthcare. 2. A web interface posts the query that was sent to the program. 3. The application integration platform, Node.js Express, receives the query via application logic. 4. After receiving the query, Node.js forwards it to Watson's Natural Language Classifier service for classification. 5. Watson Natural Language Classifier service provides a response with the top class according to the inquiry category. 6. Node.js responds to the application logic with a Natural Language Classifier answer. 7. The web interface page to be presented is determined by application logic based on the query category returned from the Watson Natural Language Classifier service. 8. The web interface shows the page with the response to the query.
The following stage is to depict the structure of models that process text data and to define interaction
pipelines with these models. To do this, think of deep DeepQA.The following elements make up the
Minimum DeepQA pipeline, which takes a question as input and outputs a response along with a
corresponding confidence score [
Consider another category of applications, namely the process of translation. Represent technology
stack for interaction with IBM BlueMix cloud [
Consider the pipeline of the language-translation process based on the represented technology stack [
Described technologies, methods, and practical approaches represent a pipeline of performing some
tasks related to text processing operations. However, known industry systems do not open for cloud
developer algorithms and approaches to text processing [
To propose conceptual basics of text processing that are composed of analytical representation of tasks and approaches to text processing description. Provide the possibility to proceed with large amounts of data. Satisfy the criteria of quick assess to text data.
Trie is a tree for storing strings that has one node for each prefix that is often used. Additional leaf nodes are used to store the strings. A trie can be compared to an m-ary tree, where m is the total number of alphabetic letters [10].
An m-way branch is used to follow the proper path in the trie, starting at the root, in order to do a search by looking at the key one character at a time. For each letter of the alphabet, a possible child exists at each node in the multi-way trie (Fig. 3). The three words BE, BED, and BACCALAUREATE are contained in the multi-way trie shown below [11].
Because they expressly presume that the keys are a sequence of values across a certain (finite) alphabet rather than a single indivisible item, tries differ from other data structures. The processing of variable-length keys is thus a situation where attempts excel. Because similar prefixes of words are concatenated, attempts, when done properly, can also enable compression of the set represented because words with the same prefix follow the same search route in the trie [12]. Trie [10] had utilized a list of the top 31 English terms to demonstrate his point [11].
Consider a general concept of knowledge processing that is presented in natural language texts. Represent it in a notation of UML sequence diagrams.
1. Preliminary analysis of the natural language text is carried out using the defined rules of facts extracted from the text. The knowledge formed from the information about the entities and relationships between them is extracted from the text (messages 1.1 and 1.2 on Fig. 6. The purpose of this step is to obtain preliminary information for an approximate estimation of the relevance of the subject of the text to the themes of the classifier stored in the ontology repository. 2. The correspondence of the subject of the text to the classifier themes is set. This operation is provided by means of matching subjects of the repository of ontologies (message 2 on Fig. 6) to text entities. The ontology classifier is presented in the form of a tree, the branches of which are the themes and subtopics of the ontology repository. 3. If the subject of the text corresponds to the subject of the classifier, then the ontology of the text is built using the knowledge that was extracted from the text earlier (messages 3.1 and 3.2 on Fig. 6). The ontology of the text is stored in the structure of a "growing pyramidal network". (Message 3.3 on fig. 6). 4. The ontology, which is transformed into a growing pyramidal network (message 4.1 on fig. 6), is stored in the repository of ontologies (message 4.2 on fig. 6). However, it can be stored as a separate repository unit or combined with other ontologies, or certain components can be used to refine existing ontologies. Note that this preserves the traces of tracing the components of the ontology with the source text.
Text1 5. Another natural language text (denoted as Text2 in Fig. 6), which is the subject of processing, is considered. Information about the entities and connections (messages 5.1 and 5.2 in Fig. 6) is extracted from it using the rules of extracting knowledge from natural language texts. 6. It is determined whether the acquired knowledge corresponds to the subject of the ontology repository, by means of comparing the knowledge of the text with ones in the classifier of the ontology repository (message 6 on Fig. 6). 7. An ontology of the second text is formed, which is stored in the form of a growing pyramidal network (messages 7.1, 7.2, and 7.3 in Fig. 6). 8. In the future, possible word processing operations are performed, which are based on operations of comparing the ontologies of the repository and the processed text (obtaining answers to questions on the text, structural comparison of texts, search for the closest texts by facts, etc.). The formal representation of the task allows to prepare of a repository of ontologies Denote a tree of ontology repository by the next where n – a number of the processes in problem domain.
Denote the nested processes by the following:
W {w1, w2 ,..., wn} , w1 {w1,1, w1,2 ,..., wm,n1 } In this example process is represented as a root process.
Initial information for composing the ontology repository consists from the triple of the next sources of information that can be transformed into text using modern ways and approaches to get text information from the audio and video content is denoted by the following: Т – texts in natural languages; А – information from the audio resources; V – information from video resources, that is extracted from audio tracks.
Denote a triple of source for ontologies designing for the process of ontology tree as the number i, і=1, …, n, where n – is a number of problem domain processes as inputi . Then
inputi {Ti , Ai ,Vi} .
If some type of initial information is absent, corresponding inputi left empty.
The lower index defines the number of problem domain processes, for example Тi – natural language text corresponding to process number i of the tree.
A full set of processes, containing all arrays of initial information from sources T,A, and V is denoted by the following:
INPUT {input1, input2 ,..., inputn}.
Represent ontology of repository corresponding to process number i of repository tree by the following:
Ri {Li К , GPN R} , where LК – tracing links between growing pyramidal networks of repository and inputi ; GPN К – Growing Pyramidal Network of Repository.
Higher index R (Repository) defines the entities before performing of procedures of its filling, for example, LК ,GPN К .
A set of growing Pyramidal Networks of ontology repository is represented in the following way:
Denote a set of tracing links between ontologies and triple inputi as
GPN A
i .
Higher index A (Add) defines ontologies and Growing Pyramidal Networks that are designed in the current operation of repository filling. (For example, GPNiA , LiA ).
Denotation that starts from the letter N (New) defines the content of growing pyramidal networks after the procedure of repository ontologies filling. For example ( NLRi , NGPNiR ).
The initial information for repository of ontologies filling is represented in the following way: . Other words (1) (2) W {w1, w2 ,..., w } n INPUT {input1, input2 ,..., inputn} inputi {Ti , Ai ,Vi}, i 1,..., n R {R1, R2 ,..., Rn} Ri {LRi , GPNiR}, i 1,..., n .
Denote an operation of filling unique information as , for example adding to repository only unique information to Growing pyramidal network is denoted by the following: NGPNiR GPNiR GPN D
i .
Denote the resulting information after repository of ontologies filling by the next: W {w1, w2 ,..., wn} INPUT {input1, input2 ,..., inputn}, inputi {Ti , Ai ,Vi}, i 1,..., n NR {NR1, NR2 ,..., NRn}, NRi {NLRi , NGPNiR}, i 1,..., n NGPNiR GPNiR GPNiA , i 1,..., n (3) NLRi LRi LiA , NLRi : NLGPNiR
{Ti A , AAi ,V Ai}, i 1,..., n , (where n is the number of tree processes), including in the repository only such metamodels that do not contain duplicate information with existing ones.
The paper contains the result of research aimed to propose a conceptual basics of semantic text comparison based on facts, extracted from a text. It is proposed to use the RDF framework to support non-relational databases, provide quick access to data, and a quick procedure for storing and reading the information. The procedure of ontology designing allows us to perform the procedure of searching facts in text effectively as well as find answers to questions with a given level of accuracy. Growing Pyramidal Networks provide quick access to any entities and facts, allowing to adoption of technologies of data processing to big data. Proposed conceptual basics allow to design of a flexible architecture and software modules removing limitations of existing technologies of data processing.
Intelligent Systems and Applications 15(4) (2023) 1-14, 2023. DOI:10.5815/ijisa.2023.04.01. [9] G. Alfio, et al. Building Cognitive Applications with IBM Watson Services, Getting Started,
IBM Redbooks, volume 1, 2017. [10] K. Markov, et al. Natural Language Addressing, IBS ISC, Kyiv, Madrid, Sofia, 2015. [11] F. Pfenning, Lecture Notes on Tries, in: Principles of Imperative Computation, 2012.
http://www.cs.cmu.edu/~fp/courses/15122-f12/lectures/21-tries. [12] S. Sahni, Tries, in: Handbook of data structures and applications, D. P. Mehta, S. Sahni (Eds.),
Chapman & Hall/CRC computer & information science, 2005. [13] F. Liang, Word Hyphenation by Computer, PhD thesis, Department of Computer Science, Stanford University, Stanford, California 94305, 1983. http://www.tug.org/docs/liang/liangthesis.pdf.