=Paper=
{{Paper
|id=Vol-2392/paper17
|storemode=property
|title=Ontology-Based Model of Information Technology for E-Learning Systems
|pdfUrl=https://ceur-ws.org/Vol-2392/paper17.pdf
|volume=Vol-2392
|authors=Zhengbing Hu,Viacheslav Liskin,Sergiy Syrota,Olha Cholyshkina,Nurgul Seilovaand
|dblpUrl=https://dblp.org/rec/conf/coapsn/HuLSCS19
}}
==Ontology-Based Model of Information Technology for E-Learning Systems==
https://ceur-ws.org/Vol-2392/paper17.pdf
Ontology-Based Model of Information Technology for
E-Learning Systems
[0000-0002-6140-3351] [0000-0002-9418-0633]
Zhengbing Hu1 , Viacheslav Liskin2 ,
2 [0000-0003-0795-167X]
Sergiy Syrota , Olha Cholyshkina3 [0000-0002-0681-0413],
[0000-0003-3827-179X]
Nurgul Seilova4
1
Central China Normal University, Wuhan, China
2
National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”, Kyiv, Ukraine
3
Interregional Academy of Personal Management, Kyiv, Ukraine
4
Satbayev University, Almaty, Kazakhstan
liskinslava@gmail.com, sergiy.syrot@gmail.com
greenhelga5@gmail.com, seilova_na@mail.ru
Abstract. The article is devoted to the ontology-based model of information
technology for e-learning systems which are based on the creation of the course
ontology and the didactics of E-learning. The authors conduct an overview and
create a classification of modern information technologies in education. The ar-
ticle also describes what benefits can be obtained using the chunk-based ap-
proach to analyze students’ knowledge. A new model of interaction between the
all members of the educational process is proposed. The authors come to the
conclusion that the educational application of the ontology-based model of in-
formation technology for e-learning systems should be based on granting lec-
ture lessons with an additional motivational component, creating computer-
based support of unsupervised work, automatizing knowledge level control
tasks.
Keywords: E-learning, Information Technology, Ontology Engineering, Chunk
based, Ontology-based, Model.
1 Introduction
The fast development of information technologies (IT) and everything connected to
the IT environment is changing modern people’s life in all spheres. Education is not
an exception. The use of IT in the educational system allows improving the learning
process through the introduction of new methods and approaches not only in learn-
ing/teaching itself, but through communication, interaction between students and
teachers, and also through the evaluation of the acquired knowledge.
With the development of WEB 3.0, there has been a trend towards the use of a seman-
tic approach to the representation of knowledge in e-learning systems (eLS). In par-
ticular, the methods of ontological engineering have been widely used [1, 2]. Unfor-
tunately, these technologies are not very popular in systems of computer education,
because there is no ontology of educational disciplines. The approach based on meta-
ontology for creation of ontologies of educational disciplines has been suggested in
the previous works of the authors [3].
Developing furthers this approach this paper proposes to apply the ontology-based
model of IT to eLS which, through the use of the ontological engineering approach,
will allow to create an ontologically managed eLS. This one unlike most of the exist-
ing eLS, will provide a flexible, individualized approach to learning. Also, in this
paper, authors have reviewed modern IT in education [4-8] in order to allocate addi-
tional paths for implementation of the methods of ontological engineering.
2 Review of Modern IT in Education
Over the past two decades [9], Learning Management Systems dominated in the field
of e-learning, (e.g. Moodle, Sakai, and others). The specified LMS have modular
architecture, are universal and provide interaction between students and teachers.
They also provide a certain amount of tools aimed at creation and organization of the
content. Under these circumstances emerges the idea to reuse content and sharing
semantically the educational content between different LMSs.
In LMS, the teacher works on his courses using the Internet, and edits the teaching
materials in real time. Thanks to this, students receive a constantly updated content.
The disadvantage of LMS systems is the complexity of their content, which is high-
quality content, however, the creation of electronic lectures, practical and laboratory
tasks, test tasks requires as much effort as is required to prepare a paper textbook.
Social networks play an important role in the development of e-learning. Teachers use
social networks to create groups for informal communication, conduct consultation,
counsel and solve other organizational issues. Under these conditions, in the center of
the pedagogical process there is a student who not only becomes more autonomous in
terms of control, but is also more active in the creation of educational information and
interaction with other learners.
Wiki-technology is a technology used for creating a website that allows users to edit
any page or create new pages on a wiki site using a web browser. Wiki-technology
allows you to accumulate knowledge by presenting them in an easy-to-handle form.
At the same time, Wiki may have different volumes and thematic focus: from global
Wikipedia and electronic encyclopedias to small reference systems [10, 16].
Momentum from online institutes such as Coursera, Intuit is gaining significant popu-
larity. These contain complete courses that consist of video lectures with subtitles,
lectures, presentations, text notes, homework assignments, test and test papers, final
examinations. After the successful completion of the course, a student receives a cer-
tificate of graduation.
One of the ways to implement e-learning is to create an e-library that will allow for
the free textbooks to be posted online in the free-form. The question of filling such a
library is solved by creating a repository of educational materials. However, its disad-
vantage is the need for administration. Also, the responsibility for the quality of the
repositories lies with the author. However, if metadata is properly organized, search in
such repositories may become available to well-known search services. The natural
indicator of the quality of content in the repository is its usage index.
Based on the review of e-learning technologies, they were divided into three groups
(Figure 1): social networks, repositories and LMS.
Fig. 1. Diagram of e-learning technologies
However, the aforementioned e-learning technologies don’t provide sufficient means
to automate the creation of educational content. In particular, they provide tools for
conducting lectures, practical, laboratory and monitoring students’ knowledge, ana-
lyzing the monitoring process, but they do not have the appropriate means to create
test questions from existing content or to create an individual learning trajectory.
With the development of Web 3.0 and the spread of social networks emerged new
forms of e-learning, which are the result of the networking of students with the teach-
er and with each other, the so-called collective knowledge. Today’s education does
not stand still, but the presence of a teacher in the virtual (Internet) environment is a
prerequisite not only in terms of content creation, but also when referring to the de-
velopment of new methodological techniques [9, 15].
3 The Model of Ontological System
Different types of educational content are generally used in e-learning, such as texts,
slides in presentations, video files, sound files, quizzes, tests etc. There is a need to
link mentioned elements together and to allocate a certain “portion” of knowledge, it
will be a set of logically related learning material that can be perceived as a unit that
is generalized by the topic of learning [3]. According to the technology offered in [2],
the educational material is organized according to the meta-ontology approach. The
basic essence of such approach is the meta-ontological system.
𝑍 =< 𝑂𝑀𝑒𝑡𝑎 , 𝑃, 𝐸 > (1)
where OMeta is the ontology of the upper level (meta-ontology) which contains general
concepts and relations that do not depend on the subject domain and are common for
all courses; P is a set of ontologies of the subject domain and certain courses; E con-
stitutes the rules according to which functions the e-learning engine, which provides
the study process. Meta-ontology OMeta consists of two parts, 𝑂𝐶𝑜𝑛𝑡𝑒𝑛𝑡 is content on-
tology and 𝑂𝐷𝑖𝑑𝑎𝑐𝑡𝑖𝑐 is didactic ontology.
Review and analysis [3] of the terms in order to determine a certain set of logically
related material and forms of knowledge delivery were carried out. This allowed to
determine that for the designation of one unit of knowledge, which on one hand corre-
sponds with the formal concept, gestalt, model, image, notion, quantum [11-14, 18].
But it can be named or imagined and is more rationally referred to by the term of a
“chunk of knowledge” in analogy to [15] further referred to as a “chunk”. The chunk-
ing technique allows us to separate data into small portions of knowledge which hu-
man short-term memory can effectively process and work with. And after studding
and memorizing these portions (“chunks”) it combines them into a single whole
knowledge. Chunking technique is often used as a main method for the simplification
of big and complex data stream. The base entity of the didactic ontology is a chunk,
while content mapping, and relations conform a didactic ontology of a discipline.
𝑂𝐷𝑖𝑑𝑎𝑐𝑡𝑖𝑐 =< 𝐶, 𝐿, 𝑅 > (2)
where 𝐶 = {𝑐𝑖 } is a set of chunks which compose the didactic ontology;
𝐿𝑖 = {𝑙𝑚𝑖 } is a set of content mappings; 𝑅 = {𝑟𝑖𝑗 } is a set of relations, there are con-
sidered two types of relations 𝑅1 𝑎𝑛𝑑 𝑅2 . 𝑅1 ⊂ 𝐶 × 𝐶 are relations among chunks;
𝑅2 ⊂ 𝐶 × 𝐿 are relations among chunks and content.
Content mappings block within the framework of one discipline and varying types of
content that allows to display synchronously different types of content in frames of
one browser or even on different devices [15, 19]. It can be distinguished that there
are relations 𝑟𝑖𝑗 of different types among the chunks of subject domain. One of those
types is the “need to understand”, for example, to understand what the decision-
making process is, a student must understand what criteria, alternatives, scales and
decision-making body are. These relations form a semantic graph of discipline.
< С𝑖 , 𝑅 >, which vertices (С𝑖 ) are chunks, and the edges 𝑅 are the set of relations,
“need to understand” (Fig. 2).
Fig. 2. Semantic graph of mapping between Chunks
The vertices of this graph correspond to the chunks, and the edges show the sequence
of learning of the study material. The topological sorting of the vertices of the graph
allows to construct the initial pathway of learning (Fig. 3).
Fig. 2. Semantic graph of mapping between Chunks after topological sorting
Presentation of the didactic ontology of the discipline in the form of a graph allows to:
add new chunks to the existing ontology; combine disciplines according to their on-
tologies; separate ontologies according to their semantic graphs on the subgraphs, for
the allocation of a certain subject of discipline for studying; convey topological sort-
ing of the vertices of the graph for the formation of the sequence of learning of the
study material; find didactic errors if it is impossible to use the topological sorting for
the graph. The didactic ontology gives us an opportunity to organize interactive edu-
cational environment and create a personal learning path.
Thus, with the help of the didactic ontology of the discipline, the task of forming the
structure and sequence of studying the discipline material is solved.
In accordance to [2] the content part of meta-ontology reflects the other type of the
connections between chunks. In comparison to the traditional approaches to building
ontologies of educational disciplines, which in fact solely constructed ontology of the
subject domain, it is proposed to construct an ontology by extracting from the text the
predicates of the following form:
notation – essence – link(relation) – description,
It can be formalized to consider a chunk as a following set of chains:
𝐶: < 𝑁, 𝐸, 𝐿, 𝐷 > (3)
where 𝑁 is notation 𝐸 = {𝐸𝑖 } is a set of essences of the subject area, 𝐿 are the links
between essences and descriptions, 𝐷 = {𝐷𝑖𝑗 } is a set of descriptions.
The set of entities which the content ontology of the discipline 𝑂 𝐶𝑜𝑛𝑡𝑒𝑛𝑡 consists of,
together with the properties of these objects and the relations between them reflects
knowledge of a certain discipline.
4 Cross Discipline Connections
One of the main features of the developed model of ontology-based IT is the possibil-
ity of cross discipline referencing of chunks in e-learning courses is [2]. With the help
of didactic ontology, the ontologies of different disciplines that are build according to
a common pattern and rule allow for automatic detection of cross discipline chunks,
as well as to make references, and thus to create new cross discipline ontologies.
For example, course tests for “Algorithms and Data Structures” include the use of
mathematical chunks such as “Matrix”.
If a student is unable to successfully pass the test for the subject “Representation of
mathematical objects using data structures”, with the help of the didactic ontology it
will be clear that the given student has not mastered this chunk, and therefore has to
be urged to review not only the lectures on “Algorithms and Data Structures”, but also
certain lectures on “Algebra and Geometry”. The given example clearly demonstrates
the cross subject referencing between “Algebra and Geometry” and “Algorithms and
Data Structures” courses with the help of a chunk from the subject areas of “Matrix”.
The example that is described above is represented in Figure 4.
Fig. 3. Cross discipline connection between two disciplines “Algebra and Geometry” and “Al-
gorithms and data structures”
The connection between course objects with the chunks of another course allows to
indirectly connecting lectures, tests and methodic materials between each other.
In this case, the didactic ontology implements a chunk-oriented approach to learning
and enables the identification of cross discipline connections.
5 Ontology-Based Model of IT for eLS
In the process of teaching of any discipline, a teacher has a curriculum of the disci-
pline and a work program of credit modules.
Based on these normative documents, the teacher develops the entire course. And he
also, distinguishes the disciplines that precede the study of this course and follow
after it.
For a more explicit consideration of the use of IT for ontology-based eLS is presented
in Fig. 5 which shows the conversion graph, and in Table 1. The explanation of the
load of arcs of the graph of transitions is provided.
Fig. 5. A graph of transitions in the IT for ontology-based eLS
In Fig. 5 the following notation has been used: 𝐸𝑃 is educational program; 𝐶ℎ𝑢𝑛𝑘𝑖𝑛𝑔
is chunk selection; 𝐶𝐿 is content labels; 𝑂𝐶𝑜𝑛𝑡𝑒𝑛𝑡 is content ontology; 𝑂𝐷𝑖𝑑𝑎𝑐𝑡 is
didactic ontology; 𝐺𝑒𝑛 is quiz genretor; 𝐶𝑜𝑛𝑡𝑒𝑛𝑡 is educational materials; 𝐸𝑛𝑔𝑖𝑛𝑒 is
the engine of an ontology-based eLS; 𝐿𝑀𝑆 is learning management system.
Table 1. Transitions in the IT for the ontology-based eLS
Transition Description
𝑧1 Selection of the chunks from a work program
𝑧2 Linking chunks with labels
𝑧3 Linking labels with educational materials
𝑧4 Construction of content ontology
𝑧5 Construction of didactic ontology
𝑧6 Quiz generation
𝑧7 Linking quizzes with chunks
𝑧8 Formatting quizzes and test
𝑧9 Transferring educational material to the engine, forming refer-
ences to educational materials
𝑧10 Construction of an individual trajectory of education
Transition Description
𝑧11 Interaction of the engine of an ontology-based eLS with exist-
ing systems
Fig. 6 presents the structural diagram of the interaction in the IT for ontology-based
eLS, which consists of two parts of the manual work of the course author (teacher),
which cannot be automated and automated work done with the help of a new applica-
tion [1].
Fig. 6. Structural diagram of the interaction in the IT for Ontology-Based eLS
In the beginning, before using the system, it is necessary to select chunks from the
related disciplines which the current discipline relies on. This step is necessary for the
entrance control of knowledge on discipline, as well as to be able to identify the gaps
and passes in the student’s knowledge in the future.
In the first step for each credit module, the author of the course allocates a set of
chunks and on its basis builds a didactic ontology:
𝜃 = {𝐶ℎ𝑖 } (4)
where 𝜃 is a set of chunks for credit module; 𝐶ℎ𝑖 an i-th chunk of credit module
𝑖 = 1, … , 𝑛.
After that, the selected chunks are formalized and the didactic ontology is built, which
will then be responsible for the individual trajectory of education. To represent the
ontology, the RDF format can be used or it can be done as a table, guided by the fol-
lowing rules: unidirectional arcs and the possibility of topological sorting.
Also, in order to create educational content, the author of the course must download
the required text file and multimedia files (images, videos, mathematical formulas,
etc.) from which the educational content will be created. It then will mark up the edu-
cational web content and distribute it between the chunks. In addition, the author of
the course formulates a system of educational goals, according to Bloom’s taxonomy
[2] and establishes the relationships of correspondence between the educational goals
and the elements of educational web content.
Fig. 7. Suggested algorithm regarding the coverage of Bloom’s taxonomy by the educational
system
The situation in Bloom’s taxonomy coverage proposed by the authors of this article in
figure 7, makes it enables to construction of the educational process in such a way,
that the teacher would create problematic situations for comprehension and perception
by the students, who search for solutions to various problems through constructive
interaction during lectures, practical, and laboratory lessons. Students’ unsupervised
work on the other hand is based on e-learning which consists of two components: the
learning content and tracking the progress.
The learning process takes place on the basis of a chunk-oriented approach, with the
help of the ontologically driven e-learning engine. For each chunk links are generated
to the educational web content and are then provided to the student so that he can
study the material using LMS or independently – in the form of hyperlinks to texts,
drawings, media in a web browser [15].
The initial trajectory of learning is obtained after the topological sorting of the onto-
graph < 𝐶ℎ𝑖 , 𝑅 >.
With the help of the developed tools [1], for each chunk, a bank of test questions and
quizzes 𝑄 is generated for its content ontology.
𝑄 = ⋃𝑛𝑖=1 𝑄𝑖 (5)
where 𝑄𝑖 is a set of generated test questions and quizzes for 𝐶ℎ𝑖 , 𝑖 = 1, … , 𝑛.
In this case, each student has a vector of marks – 𝑣̅ , such as:
𝑣̅ = (𝑔1 , … , 𝑔𝑛 ) (6)
where 𝑔𝑖 ∈ {−3, −2, −1, 𝜏, 1, 2, 3}
Herewith 𝑔𝑖 ↔ 𝐶ℎ𝑖 , that means that 𝑖-th mark correspond to 𝑖-th chunk.
The initial condition of the vector of marks is uncertain, that is, 𝑣̅ = (𝜏, … , 𝜏).
In the process of learning, the student answers 𝑄𝑖 test questions in accordance with
the constructed initial learning pathway.
After each student’s answer to the question of the corresponding chunk the grade
vector is changing. If the correct or incorrect answer is made, the following options
for placing the grade 𝑔𝑖 are available, see Table 2.
Table 2. Conversions between grades
Correct answer
Initial state End state
0 𝜏
𝜏 1
1 2
Incorrect answer
Initial state End state
1 𝜏
𝜏 0
0 -1
The ontology-based system accumulates the data about the student activity during his
interaction with the eLS. That data comprises the history of content attendance and
the history of grades’ changes. Next, on the basis of received data the system provides
recommendations for the possibility of further study or the need for re-studying of
certain material related to the specific chunk (it depends on given results). Having
completed the study of the credit module, each student has his own vector of marks,
which is analyzed by system. There are three following possible outcomes [17]:
- ∀𝑔𝑖 = 2, in case when the system will move to the next credit module;
- ∃𝑔𝑖 = 𝜏, in case when the system can’t move to the next credit module, since
there remains a chunk for which the grade is undefined.
- ∃𝑔𝑖 = −1, in case when the system can’t go to the next credit module, because
there are chunk, grades for which are unsatisfactory, and re-examination of the mate-
rial is required.
With the help of the evaluation vector, an ontology-based eLS provides recommenda-
tions for further education and builds an individual trajectory of learning in accord-
ance with the given didactic part of the meta-ontology of the discipline.
Thus, the work with the content and content of the content ontology of the discipline
correspond to the process of knowledge management. Didactic ontology of the disci-
pline implements the sequence of learning, that is, the formalization of the individual
learning pathway. Ontology-based model of IT for eLS uses chunk as a base entity. It
allows to structure the content of the course and automate the creation of educational
resources with the help of content ontology and allows to formalize the individual
trajectory of training with the help of didactic ontology.
6 Conclusion
In this paper an overview and classification of modern IT in education has been con-
ducted. These were divided into three groups: social networks, repositories and LMS.
There was developed an ontology-based model of IT for eLS, in which the basic prin-
ciples of the educational methodology are implemented. It is proposed to redistribute
the load in accordance with educational goals according to Bloom’s taxonomy,
providing class lectures with the motivational component, and using an eLS for inde-
pendent work of students, automation of their knowledge control that will increase the
time for the individual communication between the teacher and the student and the
student’s personal development.
The developed ontology-based model of IT for eLS will allow the teacher to free time
from routine work in favor of its creative component through the allocation of chunks
and then linking them with content and partial automation of the creation of test ques-
tions and computational tasks for eLS, as well as motivating students to increase the
success; in teaching students through the formation of an individual learning trajecto-
ry and the provision of new interactive learning services.
References
1. Syrota, S.V., Liskin, V.O.: Development of the test generator for “Moodle” based on
ontology. Eastern-European Journal of Enterprise Technologies, 5/2 (77), 44–48 (2015).
2. Liskin, V., Syrota S.: E-learning Information Technology Based on an Ontlogy Driven
Learning Engine, International Journal of Computer Science and Information Security.
Vol. 15(8). P. 258–263 (2007).
3. Syrota, S., Liskin, V., Kopychko, S.: Information Technology Based on Chunk Approach
for Ontology Driven E-Learning Engine, In: First International Conference on System
Analysis & Intelligent Computing (SAIC), Kyiv, pp. 1-4 (2018).
doi: 10.1109/SAIC.2018.8516862, https://ieeexplore.ieee.org/document/8516862
4. Korzh, R., Peleshchyshyn, A., Fedushko, S., Syerov, Y.: Protection of University Infor-
mation Image from Focused Aggressive Actions. In: Szewczyk, R., Kaliczynsra, M. (eds.)
Advances in Intelligent Systems and Computing: Recent Advances in Systems, Control
and Information Technology, SCIT 2016, vol. 543, pp. 104–110. Springer, Poland (2017).
DOI: 10.1007/978-3-319-48923-0_14.
5. Korzh, R., Peleshchyshyn, A., Syerov, Yu., Fedushko, S.: University’s Information Image
as a Result of University Web Communities’ Activities. In: Shakhovska, N. (ed.) Advanc-
es in Intelligent Systems and Computing: Selected Papers from the International Confer-
ence on Computer Science and Information Technologies, CSIT 2016, vol. 512, pp. 115-
127. Springer, Ukraine (2017). DOI: 10.1007/978-3-319-45991-2_8.
6. Korobiichuk, I., Fedushko, S., Juś, A., Syerov, Y.: Methods of Determining Information
Support of Web Community User Personal Data Verification System. In: Szewczyk R.,
Zieliński C., Kaliczyńska M. (eds) Automation 2017. Advances in Intelligent Systems and
Computing, vol. 550, pp 144–150. Springer (2017). DOI: 10.1007/978-3-319-54042-9_13.
7. Korzh, R., Peleschyshyn, A., Syerov, Yu., Fedushko, S.: The cataloging of virtual com-
munities of educational thematic. Webology 11 (1), article 117 (2014).
8. Korzh, R., Fedushko, S., Peleschyshyn, A.: Methods for forming an informational image
of a higher education institution. Webology 12(2), article 140 (2015).
9. Dagger, D., O’Connor, A., Lawless, S., Walsh, E., Wade, V.: Service-Oriented E-Learning
Platforms. From Monolithic Systems to Flexible Services, IEEE Internet Computing,
Vol. 11, Issue 3, P. 28–35 (2007).
10. Gnatyuk, S., Sydorenko, V., Aleksander, M.: Unified data model for defining state critical
information infrastructure in civil aviation, In Proceedings of the 2018 IEEE 9th
International Conference on Dependable Systems, Services and Technologies, Kyiv,
Ukraine, May 24-27, pp. 37-42 (2018).
11. Gnatyuk, S.: Critical Aviation Information Systems Cybersecurity, Meeting Security
Challenges Through Data Analytics and Decision Support, NATO Science for Peace and
Security Series, D: Information and Communication Security, IOS Press Ebooks, Vol.47,
№3, рр. 308-316 (2016).
12. Ganter, B., Wille, R.: Formal Concept Analysis: Mathematical Foundations, Springer,
(1999).
13. Al Hadidi, M., Al-Azzeh J., Tkalich, O., Odarchenko, R., Gnatyuk, S., Khokhlachova, Yu.:
ZigBee, Bluetooth and Wi-Fi Complex Wireless Networks Performance Increasing, Inter-
national Journal on Communications Antenna and Propagation, Vol. 7, № 1, рр. 48-56
(2017).
14. Langacker, R. W.: Introduction to Concept, Image, and Symbol. Cognitive linguistics:
basic readings. Berlin, New-York: Mouton de Gruyter, Vol. 34. P. 29-67 (2006).
15. Miller, G.: The Magical Number Seven, Plusor Minus Two: Some Limitson Our Capacity
for Processing Information, The Psychological Review, vol. 63, p. 81-97 (1956).
16. Syrota, S., Kopychko, S., Liskin, V.: Ontology-Based Approach for E-learning Course
Creation Using Chunks. Advances in Computer Science for Engineering and Education II.
ICCSEEA 2019. Advances in Intelligent Systems and Computing, p. 648-657, vol 938.
Springer, Cham (2020).
17. Tikhomirov, A., Kinash, N., Gnatyuk, S., Trufanov, A. et al: Network Society: Aggregate
Topological Models, Communications in Computer and Information Science. Verlag:
Springer International Publ, vol. 487, рр. 415-421 (2014).
18. Danik, Yu., Hryschuk, R., Gnatyuk, S.: Synergistic effects of information and cybernetic
interaction in civil aviation, Aviation, Vol. 20, №3, рр. 137-144 (2016).
19. Gnatyuk, S., Akhmetova, J., Sydorenko, V., Polishchuk, Yu., Petryk, V.: Quantitative
Evaluation Method for Mass Media Manipulative Influence on Public Opinion, Proceed-
ings of International Conference Computational Linguistics and Intelligent Systems,
pp. 71-83 (2019).