<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Towards Cross-Lingual Transfer Conversational Agent Model</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Olena Yurchenko Yevhen Kupriianov</string-name>
          <email>Olena.Yurchenko@khpi.edu.ua</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>National Technical University “Kharkiv Polytechnic Institute”</institution>
          ,
          <addr-line>Kyrpychova str. 2, Kharkiv, 61002</addr-line>
          ,
          <country country="UA">Ukraine</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Univ Lyon</institution>
          ,
          <addr-line>Univ_Lyon 2, UR ERIC, 5 avenue Mendès France, 69676 Bron Cedex</addr-line>
          ,
          <country country="FR">France</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>In this work, the goal is to develop a model of conversation system that would be able to acquire knowledge on its own, learning in a dialogue with a person. The implementation of the project lies in an interdisciplinary area including didactics, linguistics and natural language processing, and, in our opinion, the results will have important theoretical and practical significance for each field of science. The idea of the project is creating the intelligence core for the SelfLearning Conversational Agent (SLCA) that acquires knowledge through reinforcement learning using natural didactic models in the dialogue with a person in a natural language. The solution of the set tasks lies at the intersection of artificial intelligence and machine learning methods with linguistic methods of discourse analysis and text synthesis by means of a language of meanings. As a case-study we offer to consider learning a foreign language learning due to the challenges of rapid mastering it and improving and maintaining skills in its usage which are still unsolved.</p>
      </abstract>
      <kwd-group>
        <kwd>1self-learning model</kwd>
        <kwd>didactic models</kwd>
        <kwd>natural language processing</kwd>
        <kwd>computational semantics</kwd>
        <kwd>semantic analysis</kwd>
        <kwd>conversational agents</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>2,</p>
    </sec>
    <sec id="sec-2">
      <title>Alina</title>
    </sec>
    <sec id="sec-3">
      <title>Trofimova-Herman</title>
      <p>1
and</p>
      <sec id="sec-3-1">
        <title>1. Introduction</title>
        <p>Today in communication models in natural language far from all the available potential is being
realized. Most software solutions use only a small part of the features of the natural language structure
including semantics. They often ignore semantics and discourse in terms of creation of the
communication process. In our opinion, the elaboration of linguistic support of computer systems using
artificial intelligence, capable of building and understanding texts and speech in natural language, is an
urgent task.</p>
        <p>The goal of the project is to develop a learning model of the Self-Learning Conversational Agent
(SLCA) based on an extended improved methodology for learning language and evaluating its
capabilities.</p>
        <p>A child first studies their native language in the process of natural language learning, i.e. in the
dialogue with parents. Later they accumulate knowledge (meanings) and the ways to express it in the
native language while communicating with teachers at school, friends, colleagues at work etc. After
that, while studying a foreign language (or a specific domain language) they use the created knowledge
base, giving only a different form of expression to the meanings existing in their head. That is why for
creation of the basis for SLCA training we suggest to build such a model in which the meanings of
situations would be preserved and their form of expression would be chosen according to the needs of
the user.</p>
        <p>It is suggested that solving the problem of constructing a cross-lingual transfer model enables the
SLCA intelligence core to use the general concepts (meanings) regardless of the language, culture, or
sphere of human activity, and in the process of further dialogue with a person it will be possible to learn
certain forms of meaning expression in professional, age, social and other jargons, corresponding to the
situations.</p>
        <p>The SLCA model must be bi-directional, i.e., it can both learn itself and subsequently train a human
or another machine. It is supposed to improve considerably the process of gaining knowledge, as it is
enough to confirm the correct use of an expression in a certain situation instead of repeating it regularly.</p>
        <p>Thus, it becomes possible to apply the SLCA model for any language in any sphere of human
activity: robotics, manufacturing, medicine, psychology, education, social networks, e-commerce etc.
In order to develop and practically implement such a system, it is necessary to solve the following tasks:
1. To simulate the process of understanding the meaning of a statement in the context of a natural
language by recreating situations in a dialogue in which the meaning of the same concept can change
or be ambiguous;
2. To construct an algorithm for teaching the correct choice of one or another form of expression
in each dialogical situation;
3. To transform the ‘meanings’ of situations in natural human language into an artificial
metalanguage and store them in the knowledge base as a graph model;
4. To teach the system a certain ‘natural language’ by the type of machine learning with
reinforcement, which is the closest to natural didactic methods, in our opinion;
5. To teach the system not only to formulate statements in a dialogue on a certain topic for a
certain field of knowledge in natural language, but also to replenish the memorized scenarios with
new situational contexts.</p>
        <p>This way, the solution of the set tasks will lie at the intersection of artificial intelligence and machine
learning methods with linguistic methods of discourse analysis and text synthesis by means of a
language of meanings represented in the form of a knowledge graph.</p>
        <p>We suggest testing the proposed dialogue-based learning model as well as its practical application
to be conducted in the domain of foreign language learning, where:
1. There is a lack of human and time resources for working out the multitude of possible
contextual situations;
2. It is possible to anticipate the context of the dialogue and hence use transformer models of
machine learning.</p>
        <p>Therefore, a massive users’ interest and the challenges of rapid mastering a foreign language or
improving it and maintaining skills in its usage are still unsolved.</p>
        <p>The rest of the paper is organised in the following way. The next section contains a brief overview
of approaches for dialogue system creation. The state-of-the-art findings are discussed in the ‘related
works’ section. Then we describe the fundamentals of the methodology for achieving the objectives
and underline the main ideas. In the fifth section, we present our vision together with developed
framework, and discuss the results in the final section.</p>
      </sec>
      <sec id="sec-3-2">
        <title>2. Background</title>
        <p>The need for data research arises not only in business in order to increase its profits, but also in the
society and on the part of individual citizens to understand and justify socially significant
decisionmaking. The project BI4people (https://anr.fr/Project-ANR-19-CE23-0005) is created to help in solving
those problems. The aim of BI4people is to bring the power of data analysis to the largest possible
audience. However, the challenge is that the data analysis requires expert knowledge and available
resources providing information in the special domain language. Collaborative analysis and
decisionmaking give an opportunity to achieve the goals by communicating and learning over a stakeholder
group. Social networks, quizzes, brainstorms are tools for collaborative BI. So the main idea is to collect
people in one virtual or real space and encourage them leave their comments or opinions for general
purpose. Moreover, reusing another collaborator’s results or comments makes general decision-making
too collaborative. In order to organize the collaborative session, we have to communicate in the same
language in terms of understanding terminology and peculiarities of data analytics domain. It leads to
the issue of virtual assistant creation. We think that Cross-Lingual Transfer Model is the best way to
learn domain languages and can provide a real profit towards collaborative session handling.</p>
        <p>It should be mentioned that today chatbots are already widely used in various human activities: sales,
support and marketing, etc. It is an excellent tool for handling many typical user queries, which saves
staff’s time and significantly reduces the workload of human operators by automatic elimination of
typical queries. However, there are many cases of negative experience of communication with a chatbot
and the resultant reluctance to continue interaction. In general, a person’s refusal to have a dialogue
with a chatbot can be explained by the following:
• The chatbot’s automatic imposition of response scripts and the inability to solve the problem
‘in a human way’;
• The rapid learning of ‘bad words’ rather than normal statements.</p>
        <p>As a result, we need to have a tool for justification and verification of the expressions used by a
chatbot. We can conclude that only communication with a domain expert (person to person) in each
specific situation can provide reasoning for the application of certain forms of utterances in each context
and while discussing lead to development and modification of the situations themselves as well as to
creation of the necessary ‘fine-tuning’ of the chatbot.</p>
        <p>Among the best-known examples of successful dialogue systems there is GPT-3 by OpenAI and
BERT by Google. Both are built on the Large Language Model (LLM), a natural language processing
model that uses large amounts of text data to learn to predict the next words and phrases in a given text.
These models are usually trained on large data sets such as text corpora from various sources such as
books, news articles, web pages and social media, which significantly slows down or even blocks the
process of system learning for low-resource languages, for example.</p>
        <p>In addition, although these systems are used for various tasks, such as text generation, machine
translation, text classification and answering questions, such systems are not able to speculate or to
reason by analogy, because they neither have the view of the world, nor maintain ‘a conversation about
nothing’. So, for instance, in response to the sentence ‘I learnt in a different way...’ we only get an
apology from the system, not a continuation of the human dialogue.</p>
        <p>Thus, the novelty of our system compared to the Large Language Model is the creation of a machine
learning dialogue model that takes a human dialogue in its native natural language as its template and
afterwards can be used as a simulator to teach a foreign natural language or its social or professional
dialects to a human.</p>
      </sec>
      <sec id="sec-3-3">
        <title>3. Related Works</title>
        <p>
          Despite a large amount of research in the field of chatbot development, many questions have
remained open so far. The analysis of publications on the topic [
          <xref ref-type="bibr" rid="ref1 ref2 ref3 ref4">1-4</xref>
          ] shows that current trends lie in the
application of deep reinforcement learning. Compared to other methods, solutions based on
reinforcement learning make it possible to increase the flexibility and adaptability of the dialogue [
          <xref ref-type="bibr" rid="ref5 ref6">5,
6</xref>
          ], eliminate unwanted or offensive language [
          <xref ref-type="bibr" rid="ref1 ref2">1, 2</xref>
          ], maintain engagement [
          <xref ref-type="bibr" rid="ref7 ref8">7, 8</xref>
          ] etc. As the data of the
majority of studies show, for example [
          <xref ref-type="bibr" rid="ref6 ref8 ref9">6, 8, 9</xref>
          ], quite good results are achieved, however, they all use
pre-prepared large corpora of dialogues. The need for a large text corpus for training is a strong
constraint for the implementation of models for low-resource languages or specific domains.
        </p>
        <p>
          In order not to be limited by the resources of a natural language, we suggest a hybrid combination
of unsupervised cross-lingual representational learning [
          <xref ref-type="bibr" rid="ref10 ref11">10, 11</xref>
          ] and supervised learning model through
a human dialogue according to a language system model [
          <xref ref-type="bibr" rid="ref12">12</xref>
          ], in which, in addition to the structure and
the substance, a subject has been introduced as an obligatory element of the natural language system
capable to identify, interpret the structural-substantial properties and elements of the system, and
perform operations with them according to certain rules and algorithms. This interpretation reflects the
properties of natural language as the means of conveying information. Such properties include, first,
substantive properties of a language because the language embodies ‘materially’ in
informationcommunication systems. A human being controlling the learning process, an already trained computer
system, a neural network or anything like that can be a subject.
        </p>
        <p>We also offer to refer to the linguistic model of ‘Meaning-Text’, which represents language as a
multilevel model of conversions of the meaning into the text and back, since this theory postulates a
multilevel model of language. That means the one in which the construction of the text, based on a
given meaning, does not occur directly, but through a series of transitions from one level of language
to another. Each level is characterized by a set of its own units and rules of representation, as well as a
set of rules for transition from this level of representation to the neighbouring ones.</p>
        <p>Priority in our work is given to the transition from the meaning to the text, i.e. the generation of new
utterances within the proposed semantic field, as we believe that a description of the text interpretation
process can be derived from discourse analysis [13]. We base on the theoretical assumption that the
number of existing semantic units (semes) is smaller than the number of words described by them).
Similarly, the number of semantic relations between semes is smaller than the number of relations
between meanings of words in a phrase. This enables us to use the model of the metalanguage of
meaning to preserve it and generate new texts [14, 15, 16].</p>
        <p>The process of a dialogue synthesis focuses on reconstructing the underlying linguistic picture of
the world, specific to each language, and pays special attention to the study of synonymic and
periphrastic means of language [17], as well as the interpretation of complex meanings through simpler
ones [18].</p>
        <p>Based on the state-of-the-art analysis we hypothesize that the advantages of the proposed approach
will be:
•</p>
        <p>Faster process of learning, unlike, for example, corpus models that require time to create
large corpora of texts.</p>
        <p>Easier ‘fine-tuning’ to a particular language or domain compared to global language models
like GPT-3 or BERT, which will provide greater performance.</p>
        <p>A clearer representation of the same meaning, existing as concepts in many languages,
generated in different forms of utterances, which is characteristic of the natural speech we
aim to recreate.</p>
        <p>Reducing the number of examples which are needed to train the model by using transfer
learning.
•
•
•</p>
      </sec>
      <sec id="sec-3-4">
        <title>4. Methods and Materials</title>
        <p>The peculiarity of the proposed teaching model is that it is self-studying in the process of a dialogue
and is based on the experience of natural human learning of a foreign language (or a new professional
domain). Let us consider the case study of learning a foreign language. We should emphasize that, for
example, a 7-11 year old child, whose vocabulary has already been formed and is about 30 thousand
lexical units, when learning a foreign language in a foreign language environment usually shows
exponential growth during the first 6-10 months. Moreover, the smaller the child is, the easier the
learning. Everyone starts speaking in the target language starting from 3d-6th month of being in a
constant foreign dialogue, except adults who have already formed notions (concepts) and connections
between them in another language. Therefore, the learning process of adults under the same conditions
is much slower and depends more on the conscious desire and the time devoted to learning. However,
if an adult is sufficiently motivated and open to learning and they cannot be around the teacher all the
time to have a continuous dialogue, then this person requires a special tool like virtual dialogue assistant.
This way it becomes an immediate task for self-learning conversational systems.</p>
        <p>4.1.</p>
      </sec>
      <sec id="sec-3-5">
        <title>Didactic approaches to natural language teaching</title>
        <p>Taking human learning in a natural foreign language as an example, among the many
methodological approaches, two main ones should be highlighted, on which the work of our
conversational model will be based:
1. Communicative (speech-based) learning;
2. Learning based on speech models (situations).</p>
        <p>In doing so, we must consider the peculiarities of the discourse in which the communication takes
place, as well as the peculiarities of the language used.</p>
        <p>By discourse we mean speech as well as other processes of speech activity, such as text or dialogue.
On this basis, by ‘meanings’ we mean the actual realisation of the meanings of words and expressions
in a speech context, fixed at the level of language.</p>
        <p>Visually, this relationship can be represented as a three-dimensional space in which the word
meanings, which are located in the plane XY and which convey the meaning of the lexeme only by
getting into certain contexts and interacting with each other, are inextricably linked and interacting with
each other Axis X represents the lexical meaning of the word, axis Y represents the grammatical
meaning of the word, axis Z represents the syntactics of the word (lexeme meaning).</p>
        <p>At the same time, it is obvious that the semes as the minimal elements of meanings can be embedded
in language both in the lexicon and in other levels of language, such as grammar: morphology and
syntax.</p>
        <p>Thus, understanding language as a multilevel model implies that the generation of a speech process
in a natural language based on a given meaning does not occur directly, but through a series of
transitions from one formal level of representation to another: from the morphological (grammatical
level), through the syntactic (text level) to the semantic (meaning level).</p>
        <p>Each level is characterised by a set of its own units and rules (models) of representation, as well as
a set of rules of transition from this level of representation to neighbouring ones, which are supposed
to be represented as knowledge graphs [16].</p>
        <p>Consequently, a self-studying conversational system can both store the analysed meanings derived
from human dialogue and synthesise them itself.</p>
      </sec>
      <sec id="sec-3-6">
        <title>Metalanguage as a formal model of natural language processing</title>
        <p>As a semantic parser we propose to use a model of transforming the syntactic structure of a query
into a semantic one by means of conceptual graphs, which will be stored in the knowledge base in the
form of a formalized metalanguage of meanings. The conceptual graph is a way of semantic or,
otherwise, conceptual representation of situations and knowledge in models of understanding of natural
language. The nodes of the graph are those lexical units that express categories and concepts and relate
to each other by formal and associative links. The orientation of the connections in the graph is always
directed from the concepts of a higher level of generalization to the concepts that characterize them.</p>
        <p>We define an elementary meaning as a pair of connected neighbouring nodes in a graph. Such links
are not always named, sometimes they only capture the fact of some interaction between two words
(student - institute, oak - tree). The graph consists of a set of connected elementary meanings, which
enter allowed combinations, revealing lexically active and passive valences of this or that word. The
connected part of the graph that connects the two nodes forms a subgraph.</p>
        <p>The process of creating a semantic category begins with a so-called ‘key word’ being extracted from
the core structure of the sentence and the semantic analyser is the final stage of the processing. Once
the linguistic processor is completed, the analysed information is transferred to the indexing stage and
the text synthesis algorithm is activated.</p>
        <p>Models that mimic human speech activity include grammatical correctness models, reflecting the
ability to distinguish right from wrong in the language, and functional models, revealing the ability to
relate speech content (content plan) to its form (expression plan).</p>
        <p>Let us consider an example of a functional model constructed by us based on the Meaning-Text
theory (MTT), according to which language is represented as a means for its bearers to perform two
intellectual operations:
• Expressing one’s thoughts to the others, i.e. encoding the meanings (content) embedded in
texts (text construction, synthesis);
• Understanding others’ thoughts, i.e. performing the reverse operation of extracting
meanings (content) from perceived texts (text understanding, analysis).</p>
        <p>A dialogue model can be thought of as a logical device imitating the above operations in their
simplest manifestations related exclusively to knowledge of language (vocabulary and grammar). Such
a model of some language is a complexly organised set of rules whose purely mechanical application
should ideally provide:
• A transition from a given utterance in a particular language to a formal description of its
meaning, i.e. to its semantic representation or semantic record.
• Transition from the given content, i.e. semantic representation, to the appropriate utterance
in each language.</p>
        <p>Here is an example of the generation of several synonymous phrases, one of which is, for example:
‘Orwell is confident that his activism in politics makes his works the best’. The semantic structure that
sets the overall meaning of the phrases is shown in Fig. 3.</p>
        <p>The underlined word ‘confident’ means that this element of the semantic structure is
communicatively dominant. The structure given can be realised in the following forms of phrases:
1. Orwell does not doubt the positive impact of his political activism on the quality of his writing.
2. Orwell has no doubt that the fact that he is politically active makes his works better.
3. Orwell has no doubt that his political activism has a positive effect on the quality of his writings.
4. Orwell is confident that his works are made better by his political activism.
5. That the quality of his works is improved because of his political activity is not in any doubt to</p>
        <p>Orwell.
6. Orwell is convinced that his political activities have a positive influence on his work.
7. Orwell is convinced that the quality of his work is improved by his political activism.
8. Orwell is convinced that his work is made better by his political engagement.</p>
        <p>In our opinion, following the linguistic models of the MTT reveals most fully the phenomenon of
language mastery, which implies the human ability:
1. To choose language units (not only lexical) expressing a given content.
2. To correctly combine language units with certain meanings.
3. To rephrase statements with retaining their content (semantic invariant) and thereby not only to
answer the question put, but to maintain the conversation, generating new statements within the
given semantic field.</p>
        <p>Depending on which aspect of speech activity is modelled - listening or speaking - functional models
are divided into analytic and synthetic ones. The complete analytic model of a particular language takes
as ‘input’ a segment of a text (usually not less than an utterance) and gives as ‘output’ a semantic
metalingual record (semantic representation) of it (i.e. its interpretation). The complete synthetic model
of a language, being opposite to the complete analytical model, receives a semantic record (i.e. a
representation of a fragment of meaning) as ‘input’ and as ‘output’ it gives a set of synonymous texts
in this language representing the needed meaning. Analysis and synthesis models constitute an
important component of the self-learning conversational system we are developing.
4.3.</p>
      </sec>
      <sec id="sec-3-7">
        <title>Methodology for achieving the objectives</title>
        <p>From a linguistic point of view, the following methods are proposed to solve the tasks set:
1. Discourse analysis and synthesis will be carried out from a pragmatic point of view, using the
logical-semantic approach, which implies obtaining additional information to model the
dialogue by transforming its sentences (identifying corollaries, presuppositions, synonymous
constructions, etc.).
2. The metalanguage of meanings is modelled using mathematical methods in the form of
knowledge graphs (universal concepts and the relationships between them), using linguistic
methods of simplification and paraphrasing.
3. Learning experiments are carried out using didactic methods of foreign language teaching.</p>
        <p>Thus, the implementation of the project lies in the interdisciplinary field between didactics,
linguistics, and machine processing of natural language and, in our opinion, the results will have
important theoretical and practical significance for each direction.</p>
      </sec>
      <sec id="sec-3-8">
        <title>5. Results</title>
        <p>People realise that knowledge base can help in the decision-making process, avoid repeating
mistakes and wasting time. When attempting to reuse knowledge, the lack of mechanisms to identify,
find and recover reusable knowledge plays an important role. It can be concluded that the problem of
reusage has three main causes. The first relates to the understanding of the context and the solution
applied, the second relates to the content of the knowledge, and the last relates to the entire knowledge
base system that is used. Dialogue does not exist outside a certain context. Understanding the context
is based on the depth of knowledge one possesses. This allows them to correctly interpret the statements,
follow the purpose of the conversation, maintain the topic, and make the right decisions.</p>
        <p>Conversational agents and chatbots training faces significant challenges due to the complexity of
modelling in-depth knowledge. Such knowledge is accumulated over the course of a person’s life, and
the form of its expression (speech, text) reflects their social status, education, age group, occupational
specialisation to the same extent as the characteristics of the natural language they speak.</p>
        <p>It is generally recognised that knowledge can be explicit (it can be documented) or tacit (it requires
face-to-face interaction to transfer it) and should be considered in the creation process.</p>
        <p>Knowledge graphs have been proved useful for building relationships between information sources
by connecting objects of different types. This is especially useful for synthesising information using
multimodal data, such as medical records consisting of X-rays and medical reports in the form of texts
or tables. In artificial intelligence, a knowledge graph is usually represented by a set of triples such as
subject-predicate-object or object-attribute-value. Many tools exist to obtain such triples and identify
the entities that form them.</p>
        <p>All these features must be considered for creating the intelligence core of the Self-Learning
Conversational Agent. This leads to a conceptual model of dialogue formation based on the triangle of
‘meaning’ – ‘context’ – ‘expression’ (fig. 4).</p>
        <p>The natural language construction, ‘expression’, can be defined as the representation of meaning in
a particular context. The context determines the subset of expressions that are used in each piece of
speech. Meanings are formed under the pressure of context and linguistic constructions. All this
determines the relation of textual meanings, contexts and speech expressions that are used in the
dialogue. This is a limitation of both purely linguistic approaches and machine learning techniques in
terms of conversational model creation.</p>
        <p>The dialogue exists within a context and understanding that context is crucial for building knowledge
from a conversation. To effectively gain knowledge from the context of a dialogue, the following
strategies can be used. Firstly, to build knowledge from a dialogue, it is important to pay attention to
the history of the conversation. It includes understanding what has been said before, what the current
topic is, and how the conversation has progressed over time. Another important strategy for building
knowledge from a dialogue is to identify and track key concepts and entities that are being discussed.
This includes people, places, events, and ideas that are relevant to the conversation. By tracking these
entities, you can gain a deeper understanding of the context of the dialogue and how it relates to broader
knowledge domains.</p>
        <p>Overall, to build knowledge from the context of a dialogue, it is important to pay attention to the
history of the conversation, identify and track key concepts and entities, use external knowledge
sources, and consider the social context of the dialogue. There are several techniques that can be used
to help the model reuse knowledge obtained in the dialogue, such as memory networks, knowledge
bases, reinforcement learning, and hybrid approaches. A knowledge base can be used to store relevant
information, such as facts or domain-specific knowledge.</p>
        <p>Capturing and saving the meaning of context over a dialogue requires a combination of NLP
techniques, including contextual embeddings, attention mechanisms, memory mechanisms, and
discourse analysis. By incorporating these techniques into the model architecture and training process,
the model can better capture the nuances of language and generate more relevant and coherent
responses.</p>
        <p>Here is a scheme to depict a general framework for a conversational agent (fig.5):</p>
        <p>Let us consider each step.</p>
        <p>• User Input: This is the user’s input to the conversational agent, usually in the form of natural
language text or speech.
• Natural Language Understanding: This step involves processing the user input to extract
meaning and relevant information. This typically involves techniques like named entity
recognition, part-of-speech tagging and dependency parsing.
• Dialogue State Tracking: The dialogue state tracking module uses the output of the natural
language understanding module to update the current state of the conversation. This involves
keeping track of relevant context and history of the conversation.
• Dialogue Policy: The dialogue policy module decides what the conversational agent should
say next, given the current state of the conversation. This typically involves using a machine
learning algorithm, such as a reinforcement learning algorithm or a rule-based system, to
choose the most appropriate response.
• Natural Language Generation: This step involves generating a natural language response
based on the output of the dialogue policy module. This typically involves techniques like
text generation and text-to-speech conversion.
• Response: This is the final output of the conversational agent, which is presented to the user
as a natural language text or speech response.</p>
        <p>This framework provides a basic overview of the main components of a typical conversational agent.
It is possible to build a conversational agent based on a transformer model. In fact, transformers have
been used successfully in many natural languages processing applications, including chatbots and
conversational agents. Compared to seq2seq models, transformers have several advantages, including
the ability to handle longer input sequences and a more efficient training process. However, it is
important to note that the choice of model architecture depends on the specific requirements of the
application. While transformers may be better suited for some applications, seq2seq models can also be
effective, especially when paired with reinforcement learning techniques.</p>
        <p>Transfer learning is a powerful technique for building conversational agents as it allows developers
to leverage pre-trained language models to improve the performance of their models with less training
data and computing resources. It is important to choose a pre-trained language model that is well-suited
to the task and has been trained on a large amount of high-quality data. The next step is fine-tuning the
language model. Once the language model has been fine-tuned, use it to train a dialogue generation
model, such as a seq2seq model or transformer-based model. To further improve the performance of a
conversational agent, reinforcement learning can be incorporated to optimize the dialogue generation
model over time. Reinforcement learning can be used to optimize the model’s dialogue generation over
time, based on user’s feedback.</p>
        <p>By using transfer learning to fine-tune a pre-trained language model and incorporating reinforcement
learning to optimize the dialogue generation model, a powerful conversational agent can be created
with relatively few training data and computing resources. However, it is important to note that building
effective conversational agents is a complex and ongoing process that requires careful attention to users’
needs and their feedback.</p>
      </sec>
      <sec id="sec-3-9">
        <title>6. Discussion</title>
        <p>Capturing and saving the meaning of context over a dialogue is a complex task that requires a deep
understanding of natural language processing techniques. One way to capture the meaning of context
is to use contextual word embeddings, such as ELMo, BERT, or RoBERTa. By incorporating attention
mechanisms, the model can better capture the meaning of context and generate more relevant responses.
Memory mechanisms, such as memory networks or recurrent neural networks with external memory,
allow the model to store and retrieve relevant information from previous turns in the dialogue.</p>
        <p>Discourse analysis techniques can be used to analyze the structure and flow of the dialogue and
identify patterns and themes that are relevant to the context [21]. In the context of a conversational
agent, discourse analysis can be used to analyse the structure and content of the conversation between
the user and the agent. This can help to identify patterns in the conversation, understand how the
conversation is progressing, and identify any issues or problems that may arise. Discourse analysis is
an important tool for understanding how language is used in different contexts and can be a useful tool
for building more effective and naturalistic conversational agents.</p>
        <p>Training a conversational agent requires a large amount of training data, ideally in the form of
human-generated dialogues. The amount of training data required depends on several factors, including
the complexity of the application, the number of dialogues turns, and the diversity of the conversation
topics. Collecting large amounts of high-quality training data can be a time-consuming and expensive
process, especially for specialized domains. Another limitation of training data is the potential for bias.
Finally, it is worth noting that even with large amounts of training data, conversational agents may still
struggle to handle certain types of inputs or respond appropriately in certain situations. This is because
natural language is inherently ambiguous and context-dependent, and there may be many possible
interpretations of a given input or response.</p>
      </sec>
      <sec id="sec-3-10">
        <title>7. Conclusion</title>
        <p>The dialogue does not exist outside a certain context and understanding that context is crucial for
building knowledge from the dialogue. The dialogue can be used to build knowledge. The topic of the
dialogue is a critical aspect of the context. By analysing the language and structure of the conversation,
we can infer the topic being discussed and use that information to build knowledge about the subject
matter. Dialogue often involves the discussion of different entities, such as people, places or things. By
analysing the dialogue context, we can identify the relationships between these entities and use that
information to build knowledge about how they are connected. Understanding the intents and goals of
the speakers in the dialogue can provide important context for building knowledge. Dialogue often takes
place within a specific cultural and social context. By analysing the language and structure of the
conversation, we can identify these factors and use that information to build knowledge about the
cultural and social context in which the dialogue is taking place.</p>
        <p>Thus, by analysing the context of the dialogue, we can extract a wealth of information that can be
used to build knowledge about the subject matter being discussed and the broader context in which the
conversation is taking place. This information can be used to improve dialogue systems, develop better
natural language processing algorithms, and provide more accurate and relevant information to the
users.</p>
      </sec>
      <sec id="sec-3-11">
        <title>8. Acknowledgements</title>
        <p>The research study depicted in this paper is partially funded by the French National Research Agency
(ANR), project ANR-19-CE23-0005 BI4people.</p>
      </sec>
      <sec id="sec-3-12">
        <title>9. References</title>
        <p>Fox, C. (2010). Computational Semantics, In Alexander Clark, Chris Fox, and Shalom
Lappin, editors. The Handbook of Computational Linguistics and Natural Language
Processing. Malden, MA: Wiley-Blackwell, 394-428.</p>
        <p>Fähndrich, J., Trollmann, F. (2021). AI Reading, or Automatic Semantic
Decomposition into Knowledge Graphs and Symbolic reasoning through Marker Passing.
INFORMATIK 2021. Gesellschaft für Informatik, Bonn: 891-902.</p>
        <p>Milićević, J. (2007). La paraphrase. Modélisation de la paraphrase langagière. Bern:
Peter Lang. 395 pp.</p>
        <p>Al-Thanyyan, S. S., &amp; Azmi, A. M. (2021). Automated Text Simplification. ACM
Computing Surveys.</p>
        <p>Y. Kuprianov, Semantic State Superpositions and Their Treatment in Virtual
Lexicographic Laboratory for Spanish Language Dictionary, in: Proceedings of the 1st
International Conference Computational Linguistics And Intelligent Systems, COLINS-2017,
Kharkiv, Ukraine, 2017, pp. 37–46. URL:
https://colins.in.ua/wpcontent/uploads/2017/12/COLINS-2017_conference_proceedings.pdf.</p>
        <p>Dong, Xin and Gerard de Melo. “A Robust Self-Learning Framework for
CrossLingual Text Classification.” Conference on Empirical Methods in Natural Language
Processing (2019).</p>
        <p>Heidi E. Hamilton, Deborah Tannen, Deborah Schiffrin. The Handbook of Discourse
Analysis. John Wiley &amp; Sons, Mar 30, 2015. Language Arts &amp; Disciplines. 976 pages</p>
      </sec>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <surname>Cai</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          , Han,
          <string-name>
            <given-names>D.</given-names>
            ,
            <surname>Li</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            ,
            <surname>Zheng</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Z.</given-names>
            , &amp;
            <surname>Crespi</surname>
          </string-name>
          ,
          <string-name>
            <surname>N.</surname>
          </string-name>
          (
          <year>2022</year>
          ).
          <article-title>An reinforcement learning-based speech censorship chatbot system</article-title>
          .
          <source>Journal of Supercomputing</source>
          ,
          <volume>78</volume>
          (
          <issue>6</issue>
          ),
          <fpage>8751</fpage>
          -
          <lpage>8773</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <surname>Chai</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Liu</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          (
          <year>2018</year>
          ).
          <article-title>Utterance censorship of online reinforcement learning chatbot</article-title>
          .
          <source>In Proceedings - International Conference on Tools with Artificial Intelligence</source>
          ,
          <source>ICTAI</source>
          (Vol.
          <fpage>2018</fpage>
          -November, pp.
          <fpage>358</fpage>
          -
          <lpage>362</lpage>
          ). IEEE Computer Society.
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>Rajamalli</given-names>
            <surname>Keerthana</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            ,
            <surname>Fathima</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G.</given-names>
            , &amp;
            <surname>Florence</surname>
          </string-name>
          ,
          <string-name>
            <surname>L.</surname>
          </string-name>
          (
          <year>2021</year>
          ).
          <article-title>Evaluating the performance of various deep reinforcement learning algorithms for a conversational chatbot</article-title>
          .
          <source>In 2021 2nd International Conference for Emerging Technology</source>
          ,
          <string-name>
            <surname>INCET</surname>
          </string-name>
          <year>2021</year>
          .
          <article-title>Institute of Electrical and Electronics Engineers Inc</article-title>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <surname>Kumar</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Ali</surname>
            ,
            <given-names>M. M.</given-names>
          </string-name>
          (
          <year>2020</year>
          ).
          <article-title>A Review on Chatbot Design and Implementation Techniques</article-title>
          .
          <source>International Research Journal of Engineering and Technology</source>
          ,
          <volume>2791</volume>
          -
          <fpage>2800</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <surname>Yin</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Zhang</surname>
          </string-name>
          , R.,
          <string-name>
            <surname>Qi</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sun</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Tan</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          (
          <year>2018</year>
          ).
          <article-title>Context-uncertainty-aware chatbot action selection via parameterized auxiliary reinforcement learning</article-title>
          .
          <source>In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)</source>
          (Vol.
          <volume>10937</volume>
          LNAI, pp.
          <fpage>500</fpage>
          -
          <lpage>512</lpage>
          ). Springer Verlag.
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <surname>Liu</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pan</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Luo</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          (
          <year>2020</year>
          ).
          <article-title>GoChat: Goal-oriented Chatbots with Hierarchical Reinforcement Learning</article-title>
          .
          <source>In SIGIR 2020 - Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval</source>
          (pp.
          <fpage>1793</fpage>
          -
          <lpage>1796</lpage>
          ).
          <article-title>Association for Computing Machinery</article-title>
          , Inc.
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <surname>Cuayáhuitl</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lee</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ryu</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Cho</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Choi</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Indurthi</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kim</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          (
          <year>2019</year>
          ).
          <article-title>Ensemblebased deep reinforcement learning for chatbots</article-title>
          .
          <source>Neurocomputing</source>
          ,
          <volume>366</volume>
          ,
          <fpage>118</fpage>
          -
          <lpage>130</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <surname>Le</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Siddique</surname>
            ,
            <given-names>A. B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Jamour</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Oymak</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Hristidis</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          (
          <year>2021</year>
          ).
          <article-title>Generating Predictable and Adaptive Dialog Policies in Single- And Multi-domain Goal-oriented Dialog Systems</article-title>
          .
          <source>International Journal of Semantic Computing</source>
          ,
          <volume>15</volume>
          (
          <issue>4</issue>
          ),
          <fpage>419</fpage>
          -
          <lpage>439</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <surname>Permatasari</surname>
            ,
            <given-names>D. A.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Maharani</surname>
            ,
            <given-names>D. A.</given-names>
          </string-name>
          (
          <year>2021</year>
          ).
          <article-title>Combination of Natural Language Understanding and Reinforcement Learning for Booking Bot</article-title>
          .
          <source>Journal of Electrical</source>
          , Electronic, Information, and Communication Technology,
          <volume>3</volume>
          (
          <issue>1</issue>
          ),
          <fpage>12</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <surname>Ruder</surname>
            <given-names>I. Vulić S.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Søgaard</surname>
            <given-names>A.</given-names>
          </string-name>
          <article-title>A survey of cross-lingual word embedding models</article-title>
          .
          <source>Journal of Artificial Intelligence Research</source>
          <volume>65</volume>
          (
          <year>2019</year>
          ),
          <fpage>569</fpage>
          -
          <lpage>631</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <surname>Ruder</surname>
            <given-names>S. Unsupervised</given-names>
          </string-name>
          <article-title>Cross-lingual Representation Learning</article-title>
          . URL: https://www.ruder.
          <article-title>io/unsupervised-cross-lingual-learning/#unsupervised-deep-models.</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <surname>Shirokov</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          (
          <year>2021</year>
          )
          <article-title>Language</article-title>
          . Information. System.
          <source>Kyiv: PH “Akademperiodyka”</source>
          ,
          <string-name>
            <surname>Kadmon</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          (
          <year>2001</year>
          ). Formal Pragmatics: Semantics, Pragmatics, Presupposition, and
          <string-name>
            <surname>Focus</surname>
            . Oxford: Blackwell Publishers. Blackburn,
            <given-names>P.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Bos</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          (
          <year>2005</year>
          ).
          <article-title>Representation and Inference for Natural Language: A First Course in Computational Semantics</article-title>
          , Stanford, CA: CSLI Publications.
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>