=Paper=
{{Paper
|id=Vol-3862/paper1
|storemode=property
|title=Intimacy-aware Style Control in Dialog Response Generation
|pdfUrl=https://ceur-ws.org/Vol-3862/paper1.pdf
|volume=Vol-3862
|authors=Takuto Miura,Kiyoaki Shirai,Natthawut Kertkeidkachorn
|dblpUrl=https://dblp.org/rec/conf/lacatoda/MiuraSK24
}}
==Intimacy-aware Style Control in Dialog Response Generation==
https://ceur-ws.org/Vol-3862/paper1.pdf
Intimacy-aware Style Control in Dialog Response
Generation
Takuto Miura* , Kiyoaki Shirai and Natthawut Kertkeidkachorn
Japan Advanced Institute of Science and Technology, Nomi, Ishikawa, 9231211, Japan
Abstract
One of the crucial features in developing a dialog system is the choice of an appropriate speech style. This paper
proposes a novel method for training a dialog model that can effectively control the style of a response. Specifically,
the dialog model generates responses in a polite style when the user exhibits a low level of intimacy with the
system and in a casual style when the user shows a high level of intimacy. Using a pre-trained language model
(PLM) as a base dialog model, two loss functions are proposed for fine-tuning the PLM to generate responses in
an appropriate style. One is the intimacy-aware word-level loss, which serves to ensure that the dialog model
generates a polite or casual word when the user’s level of intimacy is low or high. The other is the intimacy-aware
sentence-level loss, which functions to increase the probability of the polite style of the generated utterance when
the user’s level of intimacy is low, and vice versa. The results of both automatic and human evaluations in the
experiments demonstrate that the proposed method is more effective than the baselines in generating responses
that align with the user’s degree of intimacy. Furthermore, the proposed method exhibits comparable relevance
and fluency to the PLM, indicating that the losses for the style control do not diminish the PLM’s exceptional
capacity for generating relevant and fluent responses.
Keywords
Dialog System, Speech Style, Intimacy
1. Introduction
Dialog systems that freely chat with users on a wide range of topics have attracted a great deal of
attention in recent years [1, 2, 3]. These systems are required to have comfortable conversations with
users and build long-term friendly relationships with them [4]. Humans adjust their speech style
according to their social relationships with their partners and/or the level of intimacy they share with
their partners [5, 6, 7]. Such behavior is referred to as a “style control” hereafter. One of the style
controls is to use both polite and casual styles depending on the relationship with the partner [8, 9].
Polite styles are often used in a conversation with a boss or a teacher, while casual styles are often
employed with a friend or a life partner. The style control should be considered in all conversations,
whether between humans or between humans and dialog systems [10].
The goal of this research is to develop a dialog system that flexibly controls speech styles according
to the user. Specifically, concerning the user’s intimacy with the dialog system, a response is generated
in a polite style when the user’s level of the intimacy is low, and in a casual style when the level of the
intimacy is high. To achieve this, we propose a method to incorporate knowledge necessary for style
control by fine-tuning a dialog model based on a pre-trained language model (PLM) that is capable of
generating a variety of responses consistent with the dialog context. A new loss function for fine-tuning
a dialog model is designed so that the model generates polite or casual responses when the level of the
intimacy is low or high, where the level of the intimacy is estimated from the user’s past utterances.
The contributions of this paper are summarized as follows:
• We develop a dialog system that estimates the user’s level of the intimacy and controls the polite
and casual styles in generating responses accordingly.
The 9th Linguistic and Cognitive Approaches to Dialog Agents Workshop
*
Corresponding author.
$ s2460005@jaist.ac.jp (T. Miura); kshirai@jaist.ac.jp (K. Shirai); natt@jaist.ac.jp (N. Kertkeidkachorn)
0009-0008-1178-7407 (T. Miura)
© 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
5
• We propose an approach to incorporate knowledge for style control into an existing outstanding
PLM-based dialog model.
• We demonstrate the effectiveness of the proposed method by both automatic and manual evalua-
tions.
2. Related Work
Several methods have been developed for the generation of responses in a specific style. Niu and Bansal
defined the task of generating responses in a predefined style, such as polite or rude [11]. Gao et al.
proposed a method that shared the latent space between conversational and stylistic modeling and
developed a model that generated responses in a specified style while maintaining consistency with the
dialog context [12]. Zhu et al. extended Gao’s model so that the representation of content and style
was learned in different dimensions in latent space [13]. Zheng et al. proposed a method for automatic
construction of a dialog corpus consisting of utterances in a certain style, aiming to train a stylized
dialog model [14]. Specifically, they created a Seq2Seq model, which transforms sentences in an original
dialog corpus into ones in the specified style, using texts written in that style. Tsai et al. evaluated three
approaches to achieve both content and style fidelity: conditional learning, guided fine-tuning, and
guided decoding [15]. In conditional learning, special tokens about a style are added to the input of the
dialog model. In guided fine-tuning, a style of an utterance is classified, and the classification result is
added to the input of the dialog model. In guided decoding, the weights of the output of the decoder
are determined based on the result of the style classification model. Saha et al. proposed a multitask
learning method that predicts the speaker’s personality and intention when training a dialog model
[16]. This approach is designed to control the style following the predicted state of the speaker.
Based on the aforementioned studies on maintaining a style in response generation, more recent
methods have been developed to add the capability of style control to a well-developed existing dialog
model. Sun et al. trained a dialog model using reinforcement learning, in which responses similar to
the ground-truth response and including style-related tokens got a higher reward [17]. The similarity
between responses was measured by the cosine similarity of the sentence embeddings, while the style-
specific tokes were identified by the pre-trained classification model. Li et al. retrieved a sentence similar
to an utterance from a corpus of sentences written in a specific style and fed the retrieved sentence and
the utterance into a dialog model to generate a stylized response [18]. Since the retrieved style sentence
might be harmful to generate a response consistent with the context, they incorporated an encoder that
removed features not pertinent to the context, resulting in the extraction of style features only, into the
dialog model. This encoder was trained simultaneously with the dialog model. Yang et al. proposed loss
functions using a language model that generated sentences in the specified style and a classification
model that identified the style of a sentence for fine-tuning the PLM of the dialog model [19].
Although the above previous studies can generate natural stylized responses, they are limited in their
ability to handle a single style. In contrast, our method enables the control of multiple styles according
to the user’s mental state.
Several studies focused on the emotional state of the user during a dialog. Skowron et al. showed
that interactive expression of emotions in response to the user’s feelings can significantly contribute to
enhancing the enjoyment of the chat and the emotional connection between the user and the system
[20]. D’mello and Graesser developed an intelligent tutoring agent that responds empathetically or
motivationally according to the user’s cognitive and emotional states [21]. This interactive agent dra-
matically improved the learning efficacy of students with limited domain knowledge. Thus, controlling
the type of response of the system according to the user’s internal state exerts a considerable influence.
In this study, we deal with the user’s intimacy as the user’s internal state and the speech style as the
type of response.
6
Figure 1: Overview of proposed method
3. Proposed Method
The proposed method learns a dialog model that can adapt its style to either polite or casual by the
user’s level of intimacy, which is automatically estimated from the user’s historical utterances. Figure 1
shows an overview of the proposed method. Let us suppose that a dialog model generates a response to
a user for a given dialog context 𝑋 = {𝑆1 , 𝑈1 , · · · , 𝑆𝑛 , 𝑈𝑛 }, where a system (𝑆) and a user (𝑈 ) make
an utterance alternately. Figure 1 exemplifies the case of n = 4. The intimacy estimation model employs
the user’s previous utterances 𝑋𝑢 = {𝑈1 , · · · , 𝑈𝑛 } as input and determines whether the user’s level
of intimacy with the dialog system is high or low. The dialog model accepts the context 𝑋 and the
estimated intimacy level as input and generates the response 𝑆𝑛+1 in a casual style when the user’s
intimacy is high and in a polite style when the user’s intimacy is low.
To learn the above dialog model, we extend STYLEDGPT [19], a model that consistently generates
responses in a specified style obtained by fine-tuning a PLM that can generate versatile responses. To
avoid impairing the exceptional response generation capability of the PLM, only the loss function in fine-
tuning is modified while the architecture of the PLM remains intact. Indeed, Yang et al. demonstrated
that STYLEDGPT performed well not only in its ability to produce utterances in the specified style but
also generate relevant and fluency responses [19]. First, we provide an overview of STYLEDGPT in
subsection 3.1 and then describe the details of the proposed method in the succeeding subsections.
3.1. STYLEDGPT
STYLEDGPT employs DialoGPT [22] as a PLM and learns a model that consistently generates responses
in a specified style by fine-tuning it. DialoGPT is a Seq2Seq model based on GPT-2 [23] and has been
pre-trained with a large amount of dialog data.
Word-Level Loss First, a style language model 𝑃𝑠 (𝑇 ) is trained in advance. Using a style corpus,
𝐷𝑠𝑡𝑦𝑙𝑒 , consisting of only texts in a given style, GPT-2 is trained as an autoencoder, i.e., the same
sentence 𝑇 ∈ 𝐷𝑠𝑡𝑦𝑙𝑒 is given as input and output for fine-tuning.
Let 𝑃 (𝑌 |𝑋) be a dialog model that returns a response 𝑌 for a given dialog context 𝑋. The loss is
computed for each dialog sample (𝑋, 𝑌 ) in the training data 𝐷𝑑𝑖𝑎𝑙𝑜𝑔 . 𝑌 is a sequence of words denoted
by 𝑌 = {𝑦1 , · · · , 𝑦𝑚 }. Let pY = {𝑝𝑦1 , · · · , 𝑝𝑦𝑚 } be the distribution of the predicted probability of the
next word given by the dialog model 𝑃 (𝑌 |𝑋). Also, let p ^Y = {𝑝 ^𝑦𝑚 } be the distribution of
^𝑦1 , · · · , 𝑝
the probability of predicting the next word given by the style language model 𝑃𝑠 (𝑌 ) when the output
𝑌 of the dialog model is taken as input of the style language model. The distance between pY and p ^Y
is defined as word-level loss 𝐿𝑤 as in Eq. (1).
𝑚
def
∑︁
𝐿𝑤 = 𝑑(pY ||p
^Y ) = 𝐷𝐾𝐿 (𝑝𝑦𝑖 ||𝑝
^ 𝑦𝑖 ) (1)
𝑖=1
𝐷𝐾𝐿 is the Kullback-Leibler (KL) divergence of the two probability distributions. This loss causes pY
to approach p
^Y , i.e., the dialog model is trained to produce utterances in the specified style.
7
Sentence-Level Loss First, a style discrimination model 𝑃 (𝑆|𝑇 ) is trained in advance. It identifies
whether a sentence 𝑇 is written in a given style 𝑆. This model is trained on a dataset that consists of
𝐷𝑠𝑡𝑦𝑙𝑒 , a corpus of sequences written in the specific style, as positive samples, and 𝐷𝑑𝑖𝑎𝑙𝑜𝑔 , a general
dialog corpus, as negative samples.
The loss is computed for each dialog sample (𝑋, 𝑌 ) ∈ 𝐷𝑑𝑖𝑎𝑙𝑜𝑔 . Let 𝑌^ be a response generated by the
dialog model 𝑃 (𝑌 |𝑋) for the input 𝑋, and 𝑝(𝑆|𝑌^ ) be the probability that the style of 𝑌^ is coincident
with the style 𝑆. Then, the sentence-level loss 𝐿𝑠 is defined as in Eq. (2).
𝐿𝑠 = − log 𝑝(𝑆|𝑌^ ) (2)
This loss causes the dialog model 𝑃 (𝑌 |𝑋) to produce utterances in the style 𝑆.
Negative Log-likelihood Loss The two losses mentioned above are designed to take into account
the style of a response. Fine-tuning a model with only these losses may result in a lack of consistency
between a context and a generated response. Therefore, the negative log-likelihood loss (Eq. (3)) is also
used, which is a common loss for training a dialog model. 𝑝(𝑌 |𝑋) is the probability that the dialog
model generates a ground-truth response 𝑌 from 𝑋, where (𝑋, 𝑌 ) is a sample in 𝐷𝑑𝑖𝑎𝑙𝑜𝑔 .
𝐿𝑁 𝐿𝐿 = −𝑙𝑜𝑔 𝑝(𝑌 |𝑋) (3)
3.2. Loss for Style Control
We modify the word-level and sentence-level losses in STYLEDGPT to control the style of the response
according to the user’s level of intimacy.
First, an intimacy estimation model 𝑃 (𝐼|𝑋𝑢 ) is trained. This model predicts 𝐼, the user’s level of
intimacy with a dialog system, given the user’s past 𝑛 utterances (𝑋𝑢 ) as input. In our model, 𝐼 is
defined as either low or high. The intimacy estimation model is pre-trained using a dialog corpus
annotated with the speaker’s intimacy.
To handle both polite and casual styles in response generation, two style corpora are prepared. One
𝑝𝑜
is 𝐷𝑠𝑡𝑦𝑙𝑒 , which consists of polite-style sentences, and the other is 𝐷𝑠𝑡𝑦𝑙𝑒
𝑐𝑎 , which consists of casual-style
sentences.
Intimacy-aware Word-Level Loss First, the language models of the polite and casual styles, 𝑃𝑝𝑜 (𝑇 )
𝑝𝑜
and 𝑃𝑐𝑎 (𝑇 ), are pre-trained using the corpora 𝐷𝑠𝑡𝑦𝑙𝑒 and 𝐷𝑠𝑡𝑦𝑙𝑒
𝑐𝑎 , respectively. Next, the word-level
𝑝𝑜
loss of the polite style, 𝐿𝑤 , is computed as in Eq. (1). It evaluates how likely a response is to be
polite. Similarly, the word-level loss of the casual style, 𝐿𝑐𝑎
𝑤 , is calculated. Finally, the intimacy-aware
word-level loss, 𝐿𝑖𝑛𝑤 , is defined as the weighted sum of these two losses (Eq. (4)). 𝑝(𝐼=low|𝑋𝑢 ) and
𝑝𝑜
𝑝(𝐼=high|𝑋𝑢 ) are the weights for 𝐿𝑤 and 𝐿𝑤 , which are the probability that the user’s level of intimacy
𝑐𝑎
is low and high, respectively.
𝑑𝑒𝑓
𝐿𝑖𝑛 𝑝𝑜 𝑐𝑎
𝑤 = 𝑝(𝐼=low|𝑋𝑢 ) · 𝐿𝑤 + 𝑝(𝐼=high|𝑋𝑢 ) · 𝐿𝑤 (4)
This loss is expected to encourage the generation of more polite tokens when intimacy is low and more
casual tokens when intimacy is high.
Intimacy-aware Sentence-Level Loss First, we train a style discrimination model 𝑃 ′ (𝑆|𝑇 ) that
identifies a style 𝑆 of a sentence 𝑇 , where 𝑆 is either polite or casual. The style discrimination model is
𝑝𝑜
trained in advance on training data where utterances in 𝐷𝑠𝑡𝑦𝑙𝑒 are samples of the polite class and those
in 𝐷𝑠𝑡𝑦𝑙𝑒 are samples of the casual class.
𝑐𝑎
Let 𝑌^ be the output of the dialog model 𝑃 (𝑌 |𝑋) for a given context 𝑋. Then, the style of 𝑌^
is identified by the style discrimination model and 𝑝(𝑆=polite|𝑌^ ) and 𝑝(𝑆=casual|𝑌^ ) are obtained.
Following the sentence-level loss of STYLEDGPT, the intimacy-aware sentence-level loss, 𝐿𝑖𝑛 𝑠 , is defined
as the weighted sum of the logarithms of these probabilities, using the two probabilities 𝑝(𝐼=low|𝑋𝑢 )
and 𝑝(𝐼=high|𝑋𝑢 ) as weights (Eq. (5)).
𝑑𝑒𝑓
𝐿𝑖𝑛
𝑠 = −𝑝(𝐼 =low|𝑋𝑢 ) · log 𝑝(𝑆 =polite|𝑌 ) − 𝑝(𝐼 =high|𝑋𝑢 ) · log 𝑝(𝑆 =casual|𝑌 ) (5)
8
This loss is expected to learn the dialog model to generate utterances in the polite style when the
intimacy is low and in the casual style when the intimacy is high.
Training Objective Eq. (6) shows the total loss, which is a weighted sum of the two losses concerning
a style (𝐿𝑖𝑛
𝑤 and 𝐿𝑠 ) and a general response loss (𝐿𝑁 𝐿𝐿 ).
𝑖𝑛
𝐿 = 𝛽𝑤 · 𝐿𝑖𝑛 𝑖𝑛
𝑤 + 𝛽𝑠 · 𝐿𝑠 + 𝛽𝑁 𝐿𝐿 · 𝐿𝑁 𝐿𝐿 (6)
𝛽𝑤 , 𝛽𝑠 , and 𝛽𝑁 𝐿𝐿 are hyperparameters representing the weight of each loss.
3.3. Additional Input
In addition to incorporating the information of user’s intimacy into the loss functions, the user’s level
of intimacy is explicitly given in an input to the dialog model. Specifically, the level of intimacy is
identified by 𝑃 (𝐼|𝑋𝑢 ), and then the intimacy label is added to the input as follows:
• When 𝐼=low : context
• When 𝐼=high : context
and are special tokens indicating the low and high intimacy classes, respectively. and
are special tokens indicating the beginning and end of the dialog context. This additional input
allows the dialog model to generate responses in an appropriate style that matches the identified level
of intimacy.
3.4. Sampling and Ranking
To enhance the ability of the dialog model to generate appropriately styled utterances, the sampling-
and-rank decoding strategy [12] is employed as in STYLEDGPT. First, the dialog model generates 𝑁
candidate responses using top-𝑘 sampling. Next, a style score and a content score are calculated for
each candidate response, 𝑌𝑖 , to assess the quality of 𝑌𝑖 . The candidate responses are then re-ranked by
the weighted sum of these scores, and the response with the highest score is chosen as the final output.
The style score Score𝑠𝑡𝑦𝑙𝑒 (𝑌𝑖 ) is a weighted sum of the style probabilities of 𝑌𝑖 , as in Eq. (7). The
weights are the probabilities of the low and high intimacy predicted by the history of the user’s
utterances 𝑋𝑢 . A greater style score indicates that a response is generated in the polite (or casual) style
and the user’s level of intimacy is low (or high).
𝑑𝑒𝑓
Score𝑠𝑡𝑦𝑙𝑒 (𝑌𝑖 ) = 𝑝(𝐼=low|𝑋𝑢 ) · 𝑝(𝑆=polite|𝑌𝑖 ) + 𝑝(𝐼=high|𝑋𝑢 ) · 𝑝(𝑆=casual|𝑌𝑖 ) (7)
The content score Score𝑐𝑜𝑛𝑡𝑒𝑛𝑡 (𝑌𝑖 ) is defined as the probability that the dialog model 𝑃 (𝑌 |𝑋) outputs
the response candidate 𝑌𝑖 when the dialog context 𝑋 is an input, as shown in Eq. (8). This score evaluates
the relevance of 𝑌𝑖 to 𝑋.
𝑑𝑒𝑓
Score𝑐𝑜𝑛𝑡𝑒𝑛𝑡 (𝑌𝑖 ) = 𝑃 (𝑌𝑖 |𝑋) (8)
The final score Score(𝑌𝑖 ) is defined as Eq. (9). The hyperparameter 𝜔 determines the relative
weighting of the two scores.
𝑑𝑒𝑓
Score(𝑌𝑖 ) = (1 − 𝜔) · Score𝑠𝑡𝑦𝑙𝑒 (𝑌𝑖 ) + 𝜔 · Score𝑐𝑜𝑛𝑡𝑒𝑛𝑡 (𝑌𝑖 ) (9)
4. Experiments
4.1. Datasets
Dialog Corpus with Intimacy Level Our in-house dialog corpus annotated with intimacy labels
was used to evaluate the proposed method. This corpus consists of recorded and transcribed dialogs of
9
approximately ten minutes, conducted between two speakers. For each dialog, the intimacy labels of
each of the two speakers to his/her dialog partner are annotated on a five-point scale. The statistics
of the corpus are as follows: the number of subjects who participated in the conversations is 19, the
number of conversations is 54, and the total number of utterances is 6,984. Hereafter, we refer to this
corpus as the “Japanese Intimacy Dialog Corpus” or “JID corpus” for short.
The 54 dialogs in the JID corpus were divided into three subsets: a training set of 33 dialogs, a
validation set of 9, and a test set of 12. As mentioned in Section 3, the dialog model accepts the preceding
dialog context of the user and the system, 𝑋 = {𝑆1 , 𝑈1 , · · · , 𝑆𝑛 , 𝑈𝑛 }, as input and generates the
subsequent response 𝑆𝑛+1 as output. Hereafter, the pair of a dialog context and its corresponding
response, denoted by (𝑋, 𝑆𝑛+1 ), will be referred to as an instance of response. One speaker in the
corpus was designated as the system and the other as the user to extract a dialog context and response.
The first 𝑛×2 utterances and the next utterance in a dialog were extracted as (𝑋, 𝑆𝑛+1 ). This procedure
was then repeated, with the utterance shifted one by one, to obtain multiple instances of responses.
Finally, 4,032, 921, and 1,284 instances of responses were obtained as the training, validation, and test
data, respectively.
We also used this corpus to train an intimacy estimation model. Let 𝑋𝑢 = {𝑈1 , · · · , 𝑈𝑛 } be the user’s
utterance extracted from the dialog context 𝑋 in an instance of response, and let 𝐼 be the intimacy label
for the dialog. The intimacy label was designated as “low” when the corresponding score in the JID
corpus was 1 or 2, or “high” when the value was 3, 4, or 5. The intimacy estimation model, 𝑃 (𝐼|𝑋𝑢 ), is
a binary classification model that takes 𝑋𝑢 as input and estimates the intimacy label 𝐼. The model was
trained using samples (𝑋𝑢 , 𝐼) in the training and validation data and its performance was evaluated
using the test data.
Style Corpus Two style corpora are required to train style language models and a style discrimination
𝑝𝑜 𝑐𝑎 . The KeiCO corpus [9] was used as 𝐷 𝑝𝑜 . This corpus contains utterances
model: 𝐷𝑠𝑡𝑦𝑙𝑒 and 𝐷𝑠𝑡𝑦𝑙𝑒 𝑠𝑡𝑦𝑙𝑒
using various types of honorific expressions in Japanese. Besides, 𝐷𝑠𝑡𝑦𝑙𝑒
𝑐𝑎 was constructed by extracting
𝑝𝑜
utterances from conversations between speakers who know each other in the BTSJ corpus [24]. 𝐷𝑠𝑡𝑦𝑙𝑒
contains 10,007 utterances, while 𝐷𝑠𝑡𝑦𝑙𝑒 contains 13,351 utterances.
𝑐𝑎
𝑝𝑜
To train the polite and the casual style language model, 𝑃𝑝𝑜 (𝑇 ) and 𝑃𝑐𝑎 (𝑇 ), all utterances in 𝐷𝑠𝑡𝑦𝑙𝑒
and 𝐷𝑠𝑡𝑦𝑙𝑒 , respectively, were utilized. To train the style discrimination model 𝑃 (𝑆|𝑇 ), a total of
𝑐𝑎 ′
𝑝𝑜
23,248 utterances were used, comprising 9,957 utterances in 𝐷𝑠𝑡𝑦𝑙𝑒 and 13,301 utterances in 𝐷𝑠𝑡𝑦𝑙𝑒
𝑐𝑎 . The
remaining 100 utterances (50 utterances each) were used to evaluate the style discrimination model.
4.2. Experimental Setting
The following methods, including our proposed methods, were compared in the experiment.
• DialoGPT [22] is the dialog model based on GPT-2 [23], which has been pre-trained using a
large amount of dialog data.
• S-GPT𝑝𝑜 is a STYLEDGPT that always generates polite-style responses.
• S-GPT𝑐𝑎 is a STYLEDGPT that always generates casual-style responses.
• Rule𝑎𝑢𝑡𝑜 is a method to control the style by heuristics. A response is generated by S-GPT𝑝𝑜 when
the intimacy estimation model identifies the user’s level of intimacy as low, and by S-GPT𝑐𝑎 when
it is high.
• Rule𝑔𝑜𝑙𝑑 switches between S-GPT𝑝𝑜 and S-GPT𝑐𝑎 based on the ground-truth label of the user’s
intimacy.
• I-S-GPT𝑎𝑢𝑡𝑜 is Intimacy-aware STYLEDGPT, our proposed method.
• I-S-GPT𝑔𝑜𝑙𝑑 is our proposed method, in which the ground-truth intimacy label is used instead of
an estimate based on the intimacy estimation model.
If the performance of the intimacy estimation model is inadequate, misclassification of the level
of intimacy may prevent the learning of the stylized dialog model. To verify the effectiveness of our
approach to control the style of response in terms of intimacy, I-S-GPT𝑔𝑜𝑙𝑑 was also evaluated. It can
10
be regarded as an ideal system that always correctly estimate the user’s intimacy. In this method, in
Eq. (4) and (5), the probability of the level of intimacy was approximated by the five-point intimacy
score (𝐼𝑆) in the JID corpus as 𝑝(𝐼=low|𝑋𝑢 ) ≃ 1 − 𝐼𝑆 5 and 𝑝(𝐼=high|𝑋𝑢 ) ≃ 5 . The additional input
𝐼𝑆
described in subsection 3.3 was also given by the ground-truth intimacy score, that is, is added
when 𝐼𝑆 is between 1–2, while is added when 𝐼𝑆 is between 3–5.
A method using a Large Language Model (LLM) for style-controlled generation can be considered
as a baseline. However, when a prompt is provided to ChatGPT to guess the user’s level of intimacy
and respond in an appropriate style, the generated responses are almost always polite. Therefore,
prompting-based LLM is not included in this experiment.
4.3. Implementation Details
Style Language Model and Discrimination Model The style language models 𝑃𝑝𝑜 (𝑇 ) and 𝑃𝑐𝑎 (𝑇 )
were obtained by fine-tuning GPT-2. The architecture of the style language model consists of an
embedding layer, a transformer module, and a decoding layer of GPT-2. The pre-trained model was
japanese-gpt2-medium1 , which had been trained on a large-scale Japanese dialog dataset. The learning
rate was set to 5e−4 , the batch size to 4, and the epoch to 20. The Adam optimizer [25] was used to
fine-tune the model.
The style discrimination model 𝑃 ′ (𝑆|𝑇 ) was also obtained by fine-tuning the GPT-2 model. The
architecture of the style discrimination model consists of an embedding layer, a transformer module,
and a classification layer of GPT-2. The same pre-trained model used to train the style language model
was fine-tuned using the Adam optimizer with the same hyperparameters. The style discrimination
model was evaluated using the 100 utterances not used for training. Its accuracy was 64%.
Intimacy Estimation Model Bidirectional Encoder Representations from Transformers (BERT) [26]
was used to train the intimacy estimation model. The BERT base Japanese2 , which had been trained
on Japanese Wikipedia and Japanese CC-100, was used as a pre-trained model. This BERT model was
fine-tuned using the JID corpus. As for the hyperparameters, the learning rate was set to 5e−6 , the
batch size to 1, and the epoch to 10. The Adam optimizer was used to fine-tune the model. The accuracy
of the intimacy estimation model on the test set was 69%.
The low accuracy indicates that intimacy estimation is a difficult task. Our error analysis shows that
there are few indicative words that are highly related to the speaker’s intimacy. For example, in the
sentiment analysis task, “pleasant” and “happy” are indicative words for positive emotions, and “sad”
and “unhappy” are ones for negative emotions. However, such indicative words are rare in the intimacy
estimation task. Another possible reason for the poor performance is the lack of training data. One of
the possible directions is to apply semi-supervised learning to compensate for small amounts of labeled
data with large amounts of unlabeled data.
Dialog Model The dialog model described in subsection 4.2 was obtained by fine-tuning GPT-2. The
same pre-trained model1 used for training the style language models was utilized for fine-tuning the
dialog model. As for the hyperparameters, the learning rate was set to 1e−18 , the batch size to 1, and
the epoch to 10. The Adam optimizer was used to fine-tune the model.
The parameters 𝛽𝑤 , 𝛽𝑠 , and 𝛽𝑁 𝐿𝐿 in Eq. (6) were set to 0.45, 0.45, and 0.1, respectively. These values
were optimized on the validation data according to the StyCor criterion, which will be described in §4.4.
As for the sampling-and-rank decoding strategy, the hyperparameters were set to the same values as
those used in STYLEDGPT [19], specifically 𝑘 to 40, 𝑁 to 50, and 𝜔 in Eq. (9) to 0.5.
The length of a dialog context 𝑋 = {𝑆1 , 𝑈1 , · · · , 𝑆𝑛 , 𝑈𝑛 } was set to 8, i.e., the parameter 𝑛 was set
to 4. In the preliminary experiment to evaluate the intimacy estimation model, the accuracy of the
model was measured for different values of 𝑛. The highest accuracy was obtained when 𝑛 = 4.
1
https://huggingface.co/rinna/japanese-gpt2-medium
2
https://huggingface.co/tohoku-nlp/bert-base-japanese-v2
11
Table 1
Results of Automatic Evaluation
Relevance Diversity Style
Methods
BLEU-1 BLEU-2 ROUGE-1 ROUGE-2 ROUGE-L Dist-1 Dist-2 StyCor
DialoGPT 0.0798 0.0110 0.445 0.0617 0.0400 0.674 0.915 0.115
S-GPT𝑝𝑜 0.0927 0.0118 0.393 0.0439 0.0244 0.648 0.897 0.0700
S-GPT𝑐𝑎 0.0933 0.0128 0.392 0.0556 0.0274 0.643 0.894 0.0602
Rule𝑎𝑢𝑡𝑜 0.0727 0.0082 0.428 0.0501 0.0195 0.666 0.910 0.109
Rule𝑔𝑜𝑙𝑑 0.0739 0.0078 0.432 0.0477 0.0327 0.669 0.912 0.161
I-S-GPT𝑎𝑢𝑡𝑜 0.0894 0.0103 0.372 0.0506 0.0230 0.660 0.900 0.103
I-S-GPT𝑔𝑜𝑙𝑑 0.0715 0.0078 0.414 0.0455 0.0271 0.666 0.902 0.366
4.4. Evaluation Criteria
Both automatic and human evaluations were carried out to access responses generated by various
methods.
Automatic Evaluation In automatic evaluation, the quality of the generated responses was evaluated
from three perspectives: relevance, diversity, and style. The relevance was measured by BLEU [27] and
ROUGE [28]. Specifically, the similarity between a generated response and a ground-truth response
was evaluated using BLEU-1, BLEU-2, ROUGE-1, ROUGE-2, and ROUGE-L. The diversity was measured
by Distinct-1 (Dist-1) and Distinct-2 (Dist-2), following the experiment by Li et al. [18]. The style was
evaluated using “Style Correlation” (StyCor). The StyCor metric is defined as the correlation between
the probability of the casual style 𝑝(𝑆=casual|𝑌 ) and the ground-truth level of the intimacy3 . This
correlation is high when both the predicted probability of the casual style and the intimacy level are
high, or both are low (i.e., the probability of the polite style is high and the intimacy is low). It evaluates
the extent to which the dialog model can control the style so that it generates a response in the casual
(or polite) style when the user’s level of intimacy is high (or low).
Human Evaluation The quality of the generated responses was evaluated by human subjects. A
hundred instances of responses were randomly chosen from the test set in the JID corpus. For each
instance, responses were generated using the methods described in subsection 4.2 against the dialog
context 𝑋. The responses were then evaluated by the subjects according to the following three criteria:
• Style Control: Does the response align with the appropriate style for the relationship between the
two speakers? Annotators are also instructed to read the dialog context and guess the relationship
between the speakers.
• Relevance: Is the content of the response relevant and consistent with the context?
• Fluency: Is the response natural, fluent, and free of grammatical errors?
The quality of responses was evaluated by assigning a score of 3 (appropriate), 2 (neutral), or 1
(inappropriate) for each of the three perspectives. Ten native Japanese speakers participated in the
human evaluation. The inter-annotator agreement was measured using Fleiss’s kappa [29].
5. Results
5.1. Results of Automatic Evaluation
Table 1 shows the results of the automatic evaluation. The bold indicates the best system for each
criterion. The StyCor of our proposed method using ground-truth intimacy labels, I-S-GPT𝑔𝑜𝑙𝑑 , was
0.366. It significantly outperformed the other baseline methods. Especially, the StyCor of I-S-GPT𝑔𝑜𝑙𝑑
was much better than that of the rule-based method, Rule𝑔𝑜𝑙𝑑 , which naively altered the polite and
3
The five-scale score is normalized to values between 0 and 1.
12
Table 2
Results of Human Evaluation. * means 𝑝 < 0.05. ** means 𝑝 < 0.01.
Model Style Control Relevance Fluency
Score 𝜅 𝑝 Score 𝜅 𝑝 Score 𝜅 𝑝
DialoGPT 1.98 0.26 5e−5 ** 1.51 0.22 0.15 2.16 0.39 0.03*
S-GPT𝑝𝑜 2.08 0.18 3e−4 ** 1.50 0.23 0.12 2.32 0.39 0.64
S-GPT𝑐𝑎 2.05 0.19 1e−3 ** 1.51 0.26 0.14 2.27 0.34 0.32
Rule𝑔𝑜𝑙𝑑 2.22 0.11 0.27 1.52 0.26 0.20 2.23 0.37 0.14
I-S-GPT𝑔𝑜𝑙𝑑 2.29 0.13 – 1.62 0.28 – 2.36 0.35 –
casual style generation by heuristics. These results indicated that our proposed method was superior
at generating stylized responses based on the user’s level of intimacy. When the user’s intimacy was
estimated, however, the StyCor of I-S-GPT𝑎𝑢𝑡𝑜 was 0.103, which was better than STYLEDGPT but worse
than DialoGPT. The poor StyCor of I-S-GPT𝑎𝑢𝑡𝑜 may be due to the low accuracy (69%) of the intimacy
estimation model. It was also supported by the large difference between I-S-GPT𝑎𝑢𝑡𝑜 and I-S-GPT𝑔𝑜𝑙𝑑 .
Our proposed method is highly dependent on the performance of the intimacy estimation model.
As for the relevance, S-GPT𝑐𝑎 achieved the best BLEU, while DialogGPT achieved the best ROUGE.
Our methods I-S-GPT𝑎𝑢𝑡𝑜 and I-S-GPT𝑔𝑜𝑙𝑑 were slightly worse for BLEU and obviously worse for
ROUGE than the best system, but comparable to other baselines. As for the diversity, no significant
difference of Dist-1 and Dist-2 was observed between the methods. From these results, it was found
that the outstanding ability of the pre-trained dialog model (DialoGPT) to produce relevant and diverse
responses was not remarkably damaged by incorporating the techniques of style control. Besides, no
significant difference was found in relevance and diversity between I-S-GPT𝑎𝑢𝑡𝑜 and I-S-GPT𝑔𝑜𝑙𝑑 .
5.2. Results of Human Evaluation
The automatic evaluation revealed that the StyCor scores of the methods that automatically estimated
the level of intimacy (I-S-GPT𝑎𝑢𝑡𝑜 and Rule𝑎𝑢𝑡𝑜 ) were insufficiently high. These two methods were
excluded from the human evaluation process to reduce the burden on the annotators.
Table 2 shows the results of the human evaluation. The “Score” column indicates the average of scores
assigned by the ten annotators. The “𝜅” column represents Fleiss’s 𝜅, which indicates the agreement of
scores between annotators. We also used Welch’s test to verify whether there was a significant difference
in scores between I-S-GPT𝑔𝑜𝑙𝑑 and other methods. The “𝑝” column shows the 𝑝-value associated with
this statistical test.
Style Control The proposed method, I-S-GPT𝑔𝑜𝑙𝑑 , achieved the highest score for style control. The
𝑝-values indicated that I-S-GPT𝑔𝑜𝑙𝑑 was significantly better than the other methods, except for Rule𝑔𝑜𝑙𝑑 .
These results demonstrated that our proposed method was capable of generating responses in a more
appropriate style. Rule𝑔𝑜𝑙𝑑 was the second-best method, and both I-S-GPT𝑔𝑜𝑙𝑑 and Rule𝑔𝑜𝑙𝑑 were
designed to control the style according to the level of intimacy. This confirms the validity of our
approach to consider the user’s level of intimacy to use polite and casual styles appropriately. However,
the 𝜅 for style control was 0.13, indicating that the inter-annotator agreement was relatively low.
Relevance Although I-S-GPT𝑔𝑜𝑙𝑑 was worse than the other methods in the automatic evaluation of
relevance (as shown in Table 2), it achieved the highest score for relevance in the human evaluation.
Nevertheless, no significant difference was observed. At least, the ability of the proposed method to
generate responses relevant to the dialog context was comparable to that of the other baselines.
Fluency As with the relevance score, the average score for fluency was the highest for the proposed
method. However, a significant difference was only found between DialoGPT and I-S-GPT𝑔𝑜𝑙𝑑 . The 𝜅
for fluency was higher than that for style control and relevance, indicating that the annotators exhibited
greater consistency in evaluating the fluency of the responses.
13
Table 3
Average Time of Response Generation Per Utterance (seconds)
DialoGPT S-GPT𝑝𝑜 S-GPT𝑐𝑎 Rule𝑎𝑢𝑡𝑜 I-S-GPT𝑎𝑢𝑡𝑜
3.661 4.162 4.465 5.284 5.111
Computational Time Table 3 shows a comparison of the average time required for response generation
per utterance across all test samples. A server with NVIDIA RTX A6000 48GB is used for the time
measurements. DialoGPT exhibited the shortest generation time, followed by S-GPT, I-S-GPT, and Rule.
S-GPT takes more time than DialoGPT due to the additional sampling and ranking strategy. In addition,
I-S-GPT and Rule are slower than S-GPT because they require additional processing for the intimacy
estimation.
6. Conclusion
This paper proposed a novel method of controlling the speech style of a dialog system according to the
user’s level of intimacy with the dialog system. Based on the PLM, which was a good dialog model, two
loss functions were proposed to fine-tune it to generate responses in an appropriate style. In addition,
the special token indicating the user’s level of intimacy was added to the input of the dialog model. The
results of automatic and human evaluations demonstrated that our proposed method outperformed the
baseline for style control, indicating that the method could generate responses in a polite style when
intimacy was low and a casual style when intimacy was high.
In the experiments, the accuracy of the intimacy estimation model was low, which caused a consider-
able decrease in the performance of the dialog model that used this intimacy estimation model. In the
future, by improving the intimacy estimation model, we will enhance the style control ability of the
dialog system under conditions where the ground-truth intimacy labels are not used.
It is our position that the study will not give rise to any significant ethical concerns. Our approach only
controls speech styles according to the internal state of a user, and it does not introduce or exacerbate
any ethical or social bias in a dialog system.
References
[1] C. Khatri, A. Venkatesh, B. Hedayatnia, R. Gabriel, A. Ram, R. Prasad, Alexa prize — state of the
art in conversational AI, AI Magazine 39 (2018) 40–55. URL: https://ojs.aaai.org/aimagazine/index.
php/aimagazine/article/view/2810. doi:10.1609/aimag.v39i3.2810.
[2] R. Higashinaka, K. Funakoshi, M. Inaba, Y. Tsunomori, T. Takahashi, R. Akama, Dialogue System
Live Competition: Identifying Problems with Dialogue Systems Through Live Event, Springer
Singapore, 2021, pp. 185–199.
[3] E. Dinan, V. Logacheva, V. Malykh, A. Miller, K. Shuster, J. Urbanek, D. Kiela, A. Szlam, I. Serban,
R. Lowe, et al., The second conversational intelligence challenge (convai2), in: The NeurIPS’18
Competition: From Machine Learning to Intelligent Conversations, Springer, 2020, pp. 187–208.
[4] A. Ram, R. Prasad, C. Khatri, A. Venkatesh, R. Gabriel, Q. Liu, J. Nunn, B. Hedayatnia, M. Cheng,
A. Nagar, et al., Conversational AI: The science behind the alexa prize, arXiv preprint
arXiv:1801.03604 (2018).
[5] R. Wardhaugh, J. M. Fuller, An introduction to sociolinguistics, John Wiley & Sons, 2021.
[6] E. Hovy, Generating natural language under pragmatic constraints, Journal of Pragmatics 11 (1987)
689–719. URL: https://www.sciencedirect.com/science/article/pii/0378216687901093. doi:https:
//doi.org/10.1016/0378-2166(87)90109-3.
[7] M. Silverstein, Indexical order and the dialectics of social life, Language & Communication 23
(2003) 193–229. doi:10.1016/S0271-5309(03)00013-2.
14
[8] N. Aapakallio, Understanding Through Politeness – Translations of Japanese Honorific Speech to
Finnish and English, University of Eastern Finland, 2021.
[9] M. Liu, I. Kobayashi, Construction and validation of a Japanese honorific corpus based on systemic
functional linguistics, in: J. Sälevä, C. Lignos (Eds.), Proceedings of the Workshop on Dataset
Creation for Lower-Resourced Languages within the 13th Language Resources and Evaluation
Conference, European Language Resources Association, Marseille, France, 2022, pp. 19–26. URL:
https://aclanthology.org/2022.dclrl-1.3.
[10] Y. Kageyama, Y. Chiba, T. Nose, A. Ito, Improving user impression in spoken dialog system with
gradual speech form control, in: Proceedings of the 19th Annual SIGdial Meeting on Discourse
and Dialogue, Association for Computational Linguistics, Melbourne, Australia, 2018, pp. 235–240.
URL: https://aclanthology.org/W18-5026. doi:10.18653/v1/W18-5026.
[11] T. Niu, M. Bansal, Polite dialogue generation without parallel data, Transactions of the Association
for Computational Linguistics 6 (2018) 373–389. URL: https://aclanthology.org/Q18-1027.
[12] X. Gao, Y. Zhang, S. Lee, M. Galley, C. Brockett, J. Gao, B. Dolan, Structuring latent spaces for
stylized response generation, in: K. Inui, J. Jiang, V. Ng, X. Wan (Eds.), Proceedings of the 2019
Conference on Empirical Methods in Natural Language Processing and the 9th International Joint
Conference on Natural Language Processing (EMNLP-IJCNLP), Association for Computational
Linguistics, Hong Kong, China, 2019, pp. 1814–1823. URL: https://aclanthology.org/D19-1190.
doi:10.18653/v1/D19-1190.
[13] Q. Zhu, W. Zhang, T. Liu, W. Y. Wang, Neural stylistic response generation with disentangled
latent variables, in: Proceedings of the 59th Annual Meeting of the Association for Computational
Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1:
Long Papers), Association for Computational Linguistics", Bangkok, Thailand, 2021, pp. 4391–4401.
[14] Y. Zheng, Z. Chen, R. Zhang, S. Huang, X. Mao, M. Huang, Stylized dialogue response generation
using stylized unpaired texts, in: Proceedings of the AAAI Conference on Artificial Intelligence,
AAAI Press, Online, 2021, pp. 14558–14567.
[15] A. Tsai, S. Oraby, V. Perera, J.-Y. Kao, Y. Du, A. Narayan-Chen, T. Chung, D. Hakkani-Tur, Style
control for schema-guided natural language generation, in: A. Papangelis, P. Budzianowski,
B. Liu, E. Nouri, A. Rastogi, Y.-N. Chen (Eds.), Proceedings of the 3rd Workshop on Natural
Language Processing for Conversational AI, Association for Computational Linguistics, Online,
2021, pp. 228–242. URL: https://aclanthology.org/2021.nlp4convai-1.21. doi:10.18653/v1/2021.
nlp4convai-1.21.
[16] S. Saha, S. Das, R. Srihari, Stylistic response generation by controlling personality traits and intent,
in: B. Liu, A. Papangelis, S. Ultes, A. Rastogi, Y.-N. Chen, G. Spithourakis, E. Nouri, W. Shi (Eds.),
Proceedings of the 4th Workshop on NLP for Conversational AI, Association for Computational
Linguistics, Dublin, Ireland, 2022, pp. 197–211. URL: https://aclanthology.org/2022.nlp4convai-1.16.
doi:10.18653/v1/2022.nlp4convai-1.16.
[17] Q. Sun, C. Xu, H. Hu, Y. Wang, J. Miao, X. Geng, Y. Chen, F. Xu, D. Jiang, Stylized knowledge-
grounded dialogue generation via disentangled template rewriting, in: Proceedings of the 2022
Conference of the North American Chapter of the Association for Computational Linguistics:
Human Language Technologies, Association for Computational Linguistics, Seattle, Washington,
USA, 2022, pp. 3304–3318. doi:10.18653/v1/2022.naacl-main.241.
[18] J. Li, Z. Zhang, X. Chen, D. Zhao, R. Yan, Stylized dialogue generation with feature-guided
knowledge augmentation, in: Findings of the Association for Computational Linguistics: EMNLP
2023, Association for Computational Linguistics, Sentosa Gateway, Singapore, 2023, pp. 7144–7157.
doi:10.18653/v1/2023.findings-emnlp.475.
[19] Z. Yang, W. Wu, C. Xu, X. Liang, J. Bai, L. Wang, W. Wang, Z. Li, StyleDGPT: Stylized response
generation with pre-trained language models, in: T. Cohn, Y. He, Y. Liu (Eds.), Findings of
the Association for Computational Linguistics: EMNLP 2020, Association for Computational
Linguistics, Online, 2020, pp. 1548–1559. URL: https://aclanthology.org/2020.findings-emnlp.140.
doi:10.18653/v1/2020.findings-emnlp.140.
[20] M. Skowron, S. Rank, M. Theunis, J. Sienkiewicz, The good, the bad and the neutral: affective
15
profile in dialog system-user communication, in: Proceedings of the 4th International Conference
on Affective Computing and Intelligent Interaction - Volume Part I, ACII’11, Springer-Verlag,
Berlin, Heidelberg, 2011, p. 337–346.
[21] S. D’mello, A. Graesser, Autotutor and affective autotutor: Learning by talking with cognitively
and emotionally intelligent computers that talk back, ACM Trans. Interact. Intell. Syst. 2 (2013).
URL: https://doi.org/10.1145/2395123.2395128. doi:10.1145/2395123.2395128.
[22] Y. Zhang, S. Sun, M. Galley, Y.-C. Chen, C. Brockett, X. Gao, J. Gao, J. Liu, B. Dolan, DIALOGPT :
Large-scale generative pre-training for conversational response generation, in: A. Celikyilmaz,
T.-H. Wen (Eds.), Proceedings of the 58th Annual Meeting of the Association for Computational
Linguistics: System Demonstrations, Association for Computational Linguistics, Online, 2020, pp.
270–278. URL: https://aclanthology.org/2020.acl-demos.30. doi:10.18653/v1/2020.acl-demos.
30.
[23] A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever, et al., Language models are
unsupervised multitask learners, OpenAI blog 1 (2019) 9.
[24] M. Usami (Ed.), BTSJ-Japanese Natural Conversation Corpus with Transcripts and Recordings
(March 2021), National Institute for Japanese Language and Linguistics, Japan, 2021.
[25] D. P. Kingma, J. Ba, Adam: A method for stochastic optimization, CoRR abs/1412.6980 (2014). URL:
https://api.semanticscholar.org/CorpusID:6628106.
[26] J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, BERT: Pre-training of deep bidirectional transformers
for language understanding, in: Proceedings of the 2019 Conference of the North American Chapter
of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long
and Short Papers), Association for Computational Linguistics, Minneapolis, Minnesota, 2019, pp.
4171–4186. URL: https://aclanthology.org/N19-1423. doi:10.18653/v1/N19-1423.
[27] K. Papineni, S. Roukos, T. Ward, W.-J. Zhu, BLEU: a method for automatic evaluation of machine
translation, in: Proceedings of the 40th Annual Meeting on Association for Computational
Linguistics, ACL ’02, Association for Computational Linguistics, USA, 2002, p. 311–318. URL:
https://doi.org/10.3115/1073083.1073135. doi:10.3115/1073083.1073135.
[28] C.-Y. Lin, ROUGE: A package for automatic evaluation of summaries, in: Text Summarization
Branches Out, Association for Computational Linguistics, Barcelona, Spain, 2004, pp. 74–81. URL:
https://aclanthology.org/W04-1013.
[29] J. L. Fleiss, C. Jacob, The equivalence of weighted kappa and the intraclass correlation coefficient
as measures of reliability, Educational and Psychological Measurement 33 (1973) 613–619. URL:
https://cir.nii.ac.jp/crid/1360855569674739072. doi:10.1177/001316447303300309.
16