Introduction

Dialog systems that freely chat with users on a wide range of topics have attracted a great deal of attention in recent years [1,2,3]. These systems are required to have comfortable conversations with users and build long-term friendly relationships with them [4]. Humans adjust their speech style according to their social relationships with their partners and/or the level of intimacy they share with their partners [5,6,7]. Such behavior is referred to as a "style control" hereafter. One of the style controls is to use both polite and casual styles depending on the relationship with the partner [8,9]. Polite styles are often used in a conversation with a boss or a teacher, while casual styles are often employed with a friend or a life partner. The style control should be considered in all conversations, whether between humans or between humans and dialog systems [10].

The goal of this research is to develop a dialog system that flexibly controls speech styles according to the user. Specifically, concerning the user's intimacy with the dialog system, a response is generated in a polite style when the user's level of the intimacy is low, and in a casual style when the level of the intimacy is high. To achieve this, we propose a method to incorporate knowledge necessary for style control by fine-tuning a dialog model based on a pre-trained language model (PLM) that is capable of generating a variety of responses consistent with the dialog context. A new loss function for fine-tuning a dialog model is designed so that the model generates polite or casual responses when the level of the intimacy is low or high, where the level of the intimacy is estimated from the user's past utterances.

The contributions of this paper are summarized as follows:

• We develop a dialog system that estimates the user's level of the intimacy and controls the polite and casual styles in generating responses accordingly.

• We propose an approach to incorporate knowledge for style control into an existing outstanding PLM-based dialog model. • We demonstrate the effectiveness of the proposed method by both automatic and manual evaluations.

Related Work

Several methods have been developed for the generation of responses in a specific style. Niu and Bansal defined the task of generating responses in a predefined style, such as polite or rude [11]. Gao et al. proposed a method that shared the latent space between conversational and stylistic modeling and developed a model that generated responses in a specified style while maintaining consistency with the dialog context [12]. Zhu et al. extended Gao's model so that the representation of content and style was learned in different dimensions in latent space [13]. Zheng et al. proposed a method for automatic construction of a dialog corpus consisting of utterances in a certain style, aiming to train a stylized dialog model [14]. Specifically, they created a Seq2Seq model, which transforms sentences in an original dialog corpus into ones in the specified style, using texts written in that style. Tsai et al. evaluated three approaches to achieve both content and style fidelity: conditional learning, guided fine-tuning, and guided decoding [15]. In conditional learning, special tokens about a style are added to the input of the dialog model. In guided fine-tuning, a style of an utterance is classified, and the classification result is added to the input of the dialog model. In guided decoding, the weights of the output of the decoder are determined based on the result of the style classification model. Saha et al. proposed a multitask learning method that predicts the speaker's personality and intention when training a dialog model [16]. This approach is designed to control the style following the predicted state of the speaker.

Based on the aforementioned studies on maintaining a style in response generation, more recent methods have been developed to add the capability of style control to a well-developed existing dialog model. Sun et al. trained a dialog model using reinforcement learning, in which responses similar to the ground-truth response and including style-related tokens got a higher reward [17]. The similarity between responses was measured by the cosine similarity of the sentence embeddings, while the stylespecific tokes were identified by the pre-trained classification model. Li et al. retrieved a sentence similar to an utterance from a corpus of sentences written in a specific style and fed the retrieved sentence and the utterance into a dialog model to generate a stylized response [18]. Since the retrieved style sentence might be harmful to generate a response consistent with the context, they incorporated an encoder that removed features not pertinent to the context, resulting in the extraction of style features only, into the dialog model. This encoder was trained simultaneously with the dialog model. Yang et al. proposed loss functions using a language model that generated sentences in the specified style and a classification model that identified the style of a sentence for fine-tuning the PLM of the dialog model [19].

Although the above previous studies can generate natural stylized responses, they are limited in their ability to handle a single style. In contrast, our method enables the control of multiple styles according to the user's mental state.

Several studies focused on the emotional state of the user during a dialog. Skowron et al. showed that interactive expression of emotions in response to the user's feelings can significantly contribute to enhancing the enjoyment of the chat and the emotional connection between the user and the system [20]. D'mello and Graesser developed an intelligent tutoring agent that responds empathetically or motivationally according to the user's cognitive and emotional states [21]. This interactive agent dramatically improved the learning efficacy of students with limited domain knowledge. Thus, controlling the type of response of the system according to the user's internal state exerts a considerable influence. In this study, we deal with the user's intimacy as the user's internal state and the speech style as the type of response.

Proposed Method

The proposed method learns a dialog model that can adapt its style to either polite or casual by the user's level of intimacy, which is automatically estimated from the user's historical utterances. Figure 1 shows an overview of the proposed method. Let us suppose that a dialog model generates a response to a user for a given dialog context 𝑋 = {𝑆 1 , 𝑈 1 , • • • , 𝑆 𝑛 , 𝑈 𝑛 }, where a system (𝑆) and a user (𝑈 ) make an utterance alternately. Figure 1 exemplifies the case of n = 4. The intimacy estimation model employs the user's previous utterances 𝑋 𝑢 = {𝑈 1 , • • • , 𝑈 𝑛 } as input and determines whether the user's level of intimacy with the dialog system is high or low. The dialog model accepts the context 𝑋 and the estimated intimacy level as input and generates the response 𝑆 𝑛+1 in a casual style when the user's intimacy is high and in a polite style when the user's intimacy is low.

To learn the above dialog model, we extend STYLEDGPT [19], a model that consistently generates responses in a specified style obtained by fine-tuning a PLM that can generate versatile responses. To avoid impairing the exceptional response generation capability of the PLM, only the loss function in finetuning is modified while the architecture of the PLM remains intact. Indeed, Yang et al. demonstrated that STYLEDGPT performed well not only in its ability to produce utterances in the specified style but also generate relevant and fluency responses [19]. First, we provide an overview of STYLEDGPT in subsection 3.1 and then describe the details of the proposed method in the succeeding subsections.

STYLEDGPT

STYLEDGPT employs DialoGPT [22] as a PLM and learns a model that consistently generates responses in a specified style by fine-tuning it. DialoGPT is a Seq2Seq model based on GPT-2 [23] and has been pre-trained with a large amount of dialog data.

Word-Level Loss First, a style language model 𝑃 𝑠 (𝑇 ) is trained in advance. Using a style corpus, 𝐷 𝑠𝑡𝑦𝑙𝑒 , consisting of only texts in a given style, GPT-2 is trained as an autoencoder, i.e., the same sentence 𝑇 ∈ 𝐷 𝑠𝑡𝑦𝑙𝑒 is given as input and output for fine-tuning.

Let 𝑃 (𝑌 |𝑋) be a dialog model that returns a response 𝑌 for a given dialog context 𝑋. The loss is computed for each dialog sample (𝑋, 𝑌 ) in the training data 𝐷 𝑑𝑖𝑎𝑙𝑜𝑔 . 𝑌 is a sequence of words denoted by

𝑌 = {𝑦 1 , • • • , 𝑦 𝑚 }. Let p Y = {𝑝 𝑦 1 , • • • , 𝑝 𝑦𝑚 }

be the distribution of the predicted probability of the next word given by the dialog model 𝑃 (𝑌 |𝑋). Also, let p ^Y = {𝑝 ^𝑦1 , • • • , 𝑝 ^𝑦𝑚 } be the distribution of the probability of predicting the next word given by the style language model 𝑃 𝑠 (𝑌 ) when the output 𝑌 of the dialog model is taken as input of the style language model. The distance between p Y and p ^Y is defined as word-level loss 𝐿 𝑤 as in Eq. (1).

𝐿 𝑤 = 𝑑(p Y ||p ^Y) def = 𝑚 ∑︁ 𝑖=1 𝐷 𝐾𝐿 (𝑝 𝑦 𝑖 ||𝑝 ^𝑦𝑖 )(1)

𝐷 𝐾𝐿 is the Kullback-Leibler (KL) divergence of the two probability distributions. This loss causes p Y to approach p ^Y, i.e., the dialog model is trained to produce utterances in the specified style.

Sentence-Level Loss First, a style discrimination model 𝑃 (𝑆|𝑇 ) is trained in advance. It identifies whether a sentence 𝑇 is written in a given style 𝑆. This model is trained on a dataset that consists of 𝐷 𝑠𝑡𝑦𝑙𝑒 , a corpus of sequences written in the specific style, as positive samples, and 𝐷 𝑑𝑖𝑎𝑙𝑜𝑔 , a general dialog corpus, as negative samples.

The loss is computed for each dialog sample (𝑋, 𝑌 ) ∈ 𝐷 𝑑𝑖𝑎𝑙𝑜𝑔 . Let 𝑌 ^be a response generated by the dialog model 𝑃 (𝑌 |𝑋) for the input 𝑋, and 𝑝(𝑆|𝑌 ^) be the probability that the style of 𝑌 ^is coincident with the style 𝑆. Then, the sentence-level loss 𝐿 𝑠 is defined as in Eq. (2).

𝐿 𝑠 = − log 𝑝(𝑆|𝑌 ^)(2)

This loss causes the dialog model 𝑃 (𝑌 |𝑋) to produce utterances in the style 𝑆.

Negative Log-likelihood Loss The two losses mentioned above are designed to take into account the style of a response. Fine-tuning a model with only these losses may result in a lack of consistency between a context and a generated response. Therefore, the negative log-likelihood loss (Eq. ( 3)) is also used, which is a common loss for training a dialog model. 𝑝(𝑌 |𝑋) is the probability that the dialog model generates a ground-truth response 𝑌 from 𝑋, where (𝑋, 𝑌 ) is a sample in 𝐷 𝑑𝑖𝑎𝑙𝑜𝑔 .

𝐿 𝑁 𝐿𝐿 = −𝑙𝑜𝑔 𝑝(𝑌 |𝑋)(3)

Loss for Style Control

We modify the word-level and sentence-level losses in STYLEDGPT to control the style of the response according to the user's level of intimacy.

First, an intimacy estimation model 𝑃 (𝐼|𝑋 𝑢 ) is trained. This model predicts 𝐼, the user's level of intimacy with a dialog system, given the user's past 𝑛 utterances (𝑋 𝑢 ) as input. In our model, 𝐼 is defined as either low or high. The intimacy estimation model is pre-trained using a dialog corpus annotated with the speaker's intimacy.

To handle both polite and casual styles in response generation, two style corpora are prepared. One is 𝐷 𝑝𝑜 𝑠𝑡𝑦𝑙𝑒 , which consists of polite-style sentences, and the other is 𝐷 𝑐𝑎 𝑠𝑡𝑦𝑙𝑒 , which consists of casual-style sentences.

Intimacy-aware Word-Level Loss First, the language models of the polite and casual styles, 𝑃 𝑝𝑜 (𝑇 ) and 𝑃 𝑐𝑎 (𝑇 ), are pre-trained using the corpora 𝐷 𝑝𝑜 𝑠𝑡𝑦𝑙𝑒 and 𝐷 𝑐𝑎 𝑠𝑡𝑦𝑙𝑒 , respectively. Next, the word-level loss of the polite style, 𝐿 𝑝𝑜 𝑤 , is computed as in Eq. ( 1). It evaluates how likely a response is to be polite. Similarly, the word-level loss of the casual style, 𝐿 𝑐𝑎 𝑤 , is calculated. Finally, the intimacy-aware word-level loss, 𝐿 𝑖𝑛 𝑤 , is defined as the weighted sum of these two losses (Eq. ( 4)). 𝑝(𝐼=low|𝑋 𝑢 ) and 𝑝(𝐼=high|𝑋 𝑢 ) are the weights for 𝐿 𝑝𝑜 𝑤 and 𝐿 𝑐𝑎 𝑤 , which are the probability that the user's level of intimacy is low and high, respectively.

𝐿 𝑖𝑛 𝑤 𝑑𝑒𝑓 = 𝑝(𝐼=low|𝑋 𝑢 ) • 𝐿 𝑝𝑜 𝑤 + 𝑝(𝐼=high|𝑋 𝑢 ) • 𝐿 𝑐𝑎 𝑤 (4)

This loss is expected to encourage the generation of more polite tokens when intimacy is low and more casual tokens when intimacy is high. Following the sentence-level loss of STYLEDGPT, the intimacy-aware sentence-level loss, 𝐿 𝑖𝑛 𝑠 , is defined as the weighted sum of the logarithms of these probabilities, using the two probabilities 𝑝(𝐼=low|𝑋 𝑢 ) and 𝑝(𝐼=high|𝑋 𝑢 ) as weights (Eq. ( 5)).

Intimacy-aware𝐿 𝑖𝑛 𝑠 𝑑𝑒𝑓 = −𝑝(𝐼=low|𝑋 𝑢 ) • log 𝑝(𝑆=polite|𝑌 ) − 𝑝(𝐼=high|𝑋 𝑢 ) • log 𝑝(𝑆=casual|𝑌 )(5)

This loss is expected to learn the dialog model to generate utterances in the polite style when the intimacy is low and in the casual style when the intimacy is high.

Training Objective Eq. ( 6) shows the total loss, which is a weighted sum of the two losses concerning a style (𝐿 𝑖𝑛 𝑤 and 𝐿 𝑖𝑛 𝑠 ) and a general response loss (𝐿 𝑁 𝐿𝐿 ).

𝐿 = 𝛽 𝑤 • 𝐿 𝑖𝑛 𝑤 + 𝛽 𝑠 • 𝐿 𝑖𝑛 𝑠 + 𝛽 𝑁 𝐿𝐿 • 𝐿 𝑁 𝐿𝐿(6)

𝛽 𝑤 , 𝛽 𝑠 , and 𝛽 𝑁 𝐿𝐿 are hyperparameters representing the weight of each loss.

Additional Input

In addition to incorporating the information of user's intimacy into the loss functions, the user's level of intimacy is explicitly given in an input to the dialog model. Specifically, the level of intimacy is identified by 𝑃 (𝐼|𝑋 𝑢 ), and then the intimacy label is added to the input as follows:

• When 𝐼=low : <l> <s> context </s> • When 𝐼=high : <h> <s> context </s> <l> and <h> are special tokens indicating the low and high intimacy classes, respectively. <s> and </s> are special tokens indicating the beginning and end of the dialog context. This additional input allows the dialog model to generate responses in an appropriate style that matches the identified level of intimacy.

Sampling and Ranking

To enhance the ability of the dialog model to generate appropriately styled utterances, the samplingand-rank decoding strategy [12] is employed as in STYLEDGPT. First, the dialog model generates 𝑁 candidate responses using top-𝑘 sampling. Next, a style score and a content score are calculated for each candidate response, 𝑌 𝑖 , to assess the quality of 𝑌 𝑖 . The candidate responses are then re-ranked by the weighted sum of these scores, and the response with the highest score is chosen as the final output.

The style score Score 𝑠𝑡𝑦𝑙𝑒 (𝑌 𝑖 ) is a weighted sum of the style probabilities of 𝑌 𝑖 , as in Eq. ( 7). The weights are the probabilities of the low and high intimacy predicted by the history of the user's utterances 𝑋 𝑢 . A greater style score indicates that a response is generated in the polite (or casual) style and the user's level of intimacy is low (or high).

Score 𝑠𝑡𝑦𝑙𝑒 (𝑌 𝑖 ) 𝑑𝑒𝑓 = 𝑝(𝐼=low|𝑋 𝑢 ) • 𝑝(𝑆=polite|𝑌 𝑖 ) + 𝑝(𝐼=high|𝑋 𝑢 ) • 𝑝(𝑆=casual|𝑌 𝑖 )(7)

The content score Score 𝑐𝑜𝑛𝑡𝑒𝑛𝑡 (𝑌 𝑖 ) is defined as the probability that the dialog model 𝑃 (𝑌 |𝑋) outputs the response candidate 𝑌 𝑖 when the dialog context 𝑋 is an input, as shown in Eq. ( 8). This score evaluates the relevance of 𝑌 𝑖 to 𝑋.

Score 𝑐𝑜𝑛𝑡𝑒𝑛𝑡 (𝑌 𝑖 ) 𝑑𝑒𝑓 = 𝑃 (𝑌 𝑖 |𝑋)(8)

The final score Score(𝑌 𝑖 ) is defined as Eq. ( 9). The hyperparameter 𝜔 determines the relative weighting of the two scores.

Score(𝑌𝑖 ) 𝑑𝑒𝑓 = (1 − 𝜔) • Score 𝑠𝑡𝑦𝑙𝑒 (𝑌 𝑖 ) + 𝜔 • Score 𝑐𝑜𝑛𝑡𝑒𝑛𝑡 (𝑌 𝑖 )(9)

4. Experiments

Datasets

Dialog Corpus with Intimacy Level Our in-house dialog corpus annotated with intimacy labels was used to evaluate the proposed method. This corpus consists of recorded and transcribed dialogs of approximately ten minutes, conducted between two speakers. For each dialog, the intimacy labels of each of the two speakers to his/her dialog partner are annotated on a five-point scale. The statistics of the corpus are as follows: the number of subjects who participated in the conversations is 19, the number of conversations is 54, and the total number of utterances is 6,984. Hereafter, we refer to this corpus as the "Japanese Intimacy Dialog Corpus" or "JID corpus" for short. The 54 dialogs in the JID corpus were divided into three subsets: a training set of 33 dialogs, a validation set of 9, and a test set of 12. As mentioned in Section 3, the dialog model accepts the preceding dialog context of the user and the system, 𝑋 = {𝑆 1 , 𝑈 1 , • • • , 𝑆 𝑛 , 𝑈 𝑛 }, as input and generates the subsequent response 𝑆 𝑛+1 as output. Hereafter, the pair of a dialog context and its corresponding response, denoted by (𝑋, 𝑆 𝑛+1 ), will be referred to as an instance of response. One speaker in the corpus was designated as the system and the other as the user to extract a dialog context and response. The first 𝑛×2 utterances and the next utterance in a dialog were extracted as (𝑋, 𝑆 𝑛+1 ). This procedure was then repeated, with the utterance shifted one by one, to obtain multiple instances of responses. Finally, 4,032, 921, and 1,284 instances of responses were obtained as the training, validation, and test data, respectively.

We also used this corpus to train an intimacy estimation model. Let 𝑋 𝑢 = {𝑈 1 , • • • , 𝑈 𝑛 } be the user's utterance extracted from the dialog context 𝑋 in an instance of response, and let 𝐼 be the intimacy label for the dialog. The intimacy label was designated as "low" when the corresponding score in the JID corpus was 1 or 2, or "high" when the value was 3, 4, or 5. The intimacy estimation model, 𝑃 (𝐼|𝑋 𝑢 ), is a binary classification model that takes 𝑋 𝑢 as input and estimates the intimacy label 𝐼. The model was trained using samples (𝑋 𝑢 , 𝐼) in the training and validation data and its performance was evaluated using the test data.

Style Corpus

Two style corpora are required to train style language models and a style discrimination model: 𝐷 𝑝𝑜 𝑠𝑡𝑦𝑙𝑒 and 𝐷 𝑐𝑎 𝑠𝑡𝑦𝑙𝑒 . The KeiCO corpus [9] was used as 𝐷 𝑝𝑜 𝑠𝑡𝑦𝑙𝑒 . This corpus contains utterances using various types of honorific expressions in Japanese. Besides, 𝐷 𝑐𝑎 𝑠𝑡𝑦𝑙𝑒 was constructed by extracting utterances from conversations between speakers who know each other in the BTSJ corpus [24]. 𝐷 𝑝𝑜 𝑠𝑡𝑦𝑙𝑒 contains 10,007 utterances, while 𝐷 𝑐𝑎 𝑠𝑡𝑦𝑙𝑒 contains 13,351 utterances. To train the polite and the casual style language model, 𝑃 𝑝𝑜 (𝑇 ) and 𝑃 𝑐𝑎 (𝑇 ), all utterances in 𝐷 𝑝𝑜 𝑠𝑡𝑦𝑙𝑒 and 𝐷 𝑐𝑎 𝑠𝑡𝑦𝑙𝑒 , respectively, were utilized. To train the style discrimination model 𝑃 ′ (𝑆|𝑇 ), a total of 23,248 utterances were used, comprising 9,957 utterances in 𝐷 𝑝𝑜 𝑠𝑡𝑦𝑙𝑒 and 13,301 utterances in 𝐷 𝑐𝑎 𝑠𝑡𝑦𝑙𝑒 . The remaining 100 utterances (50 utterances each) were used to evaluate the style discrimination model.

Experimental Setting

The following methods, including our proposed methods, were compared in the experiment.

• DialoGPT [22] is the dialog model based on GPT-2 [23], which has been pre-trained using a large amount of dialog data. • S-GPT 𝑝𝑜 is a STYLEDGPT that always generates polite-style responses.

• S-GPT 𝑐𝑎 is a STYLEDGPT that always generates casual-style responses.

• Rule 𝑎𝑢𝑡𝑜 is a method to control the style by heuristics. A response is generated by S-GPT 𝑝𝑜 when the intimacy estimation model identifies the user's level of intimacy as low, and by S-GPT 𝑐𝑎 when it is high. • Rule 𝑔𝑜𝑙𝑑 switches between S-GPT 𝑝𝑜 and S-GPT 𝑐𝑎 based on the ground-truth label of the user's intimacy. • I-S-GPT 𝑎𝑢𝑡𝑜 is Intimacy-aware STYLEDGPT, our proposed method.

• I-S-GPT 𝑔𝑜𝑙𝑑 is our proposed method, in which the ground-truth intimacy label is used instead of an estimate based on the intimacy estimation model.

If the performance of the intimacy estimation model is inadequate, misclassification of the level of intimacy may prevent the learning of the stylized dialog model. To verify the effectiveness of our approach to control the style of response in terms of intimacy, I-S-GPT 𝑔𝑜𝑙𝑑 was also evaluated. It can be regarded as an ideal system that always correctly estimate the user's intimacy. In this method, in Eq. ( 4) and ( 5), the probability of the level of intimacy was approximated by the five-point intimacy score (𝐼𝑆) in the JID corpus as 𝑝(𝐼=low|𝑋 𝑢 ) ≃ 1 − 𝐼𝑆 5 and 𝑝(𝐼=high|𝑋 𝑢 ) ≃ 𝐼𝑆 5 . The additional input described in subsection 3.3 was also given by the ground-truth intimacy score, that is, <l> is added when 𝐼𝑆 is between 1-2, while <h> is added when 𝐼𝑆 is between 3-5.

A method using a Large Language Model (LLM) for style-controlled generation can be considered as a baseline. However, when a prompt is provided to ChatGPT to guess the user's level of intimacy and respond in an appropriate style, the generated responses are almost always polite. Therefore, prompting-based LLM is not included in this experiment.

Implementation Details

Style Language Model and Discrimination Model

The style language models 𝑃 𝑝𝑜 (𝑇 ) and 𝑃 𝑐𝑎 (𝑇 ) were obtained by fine-tuning GPT-2. The architecture of the style language model consists of an embedding layer, a transformer module, and a decoding layer of GPT-2. The pre-trained model was japanese-gpt2-medium 1 , which had been trained on a large-scale Japanese dialog dataset. The learning rate was set to 5e −4 , the batch size to 4, and the epoch to 20. The Adam optimizer [25] was used to fine-tune the model.

The style discrimination model 𝑃 ′ (𝑆|𝑇 ) was also obtained by fine-tuning the GPT-2 model. The architecture of the style discrimination model consists of an embedding layer, a transformer module, and a classification layer of GPT-2. The same pre-trained model used to train the style language model was fine-tuned using the Adam optimizer with the same hyperparameters. The style discrimination model was evaluated using the 100 utterances not used for training. Its accuracy was 64%.

Intimacy Estimation Model Bidirectional Encoder Representations from Transformers (BERT) [26] was used to train the intimacy estimation model. The BERT base Japanese2 , which had been trained on Japanese Wikipedia and Japanese CC-100, was used as a pre-trained model. This BERT model was fine-tuned using the JID corpus. As for the hyperparameters, the learning rate was set to 5e −6 , the batch size to 1, and the epoch to 10. The Adam optimizer was used to fine-tune the model. The accuracy of the intimacy estimation model on the test set was 69%.

The low accuracy indicates that intimacy estimation is a difficult task. Our error analysis shows that there are few indicative words that are highly related to the speaker's intimacy. For example, in the sentiment analysis task, "pleasant" and "happy" are indicative words for positive emotions, and "sad" and "unhappy" are ones for negative emotions. However, such indicative words are rare in the intimacy estimation task. Another possible reason for the poor performance is the lack of training data. One of the possible directions is to apply semi-supervised learning to compensate for small amounts of labeled data with large amounts of unlabeled data.

Dialog Model

The dialog model described in subsection 4.2 was obtained by fine-tuning GPT-2. The same pre-trained model 1 used for training the style language models was utilized for fine-tuning the dialog model. As for the hyperparameters, the learning rate was set to 1e −18 , the batch size to 1, and the epoch to 10. The Adam optimizer was used to fine-tune the model.

The parameters 𝛽 𝑤 , 𝛽 𝑠 , and 𝛽 𝑁 𝐿𝐿 in Eq. ( 6) were set to 0.45, 0.45, and 0.1, respectively. These values were optimized on the validation data according to the StyCor criterion, which will be described in §4.4. As for the sampling-and-rank decoding strategy, the hyperparameters were set to the same values as those used in STYLEDGPT [19], specifically 𝑘 to 40, 𝑁 to 50, and 𝜔 in Eq. (9) to 0.5.

The length of a dialog context 𝑋 = {𝑆 1 , 𝑈 1 , • • • , 𝑆 𝑛 , 𝑈 𝑛 } was set to 8, i.e., the parameter 𝑛 was set to 4. In the preliminary experiment to evaluate the intimacy estimation model, the accuracy of the model was measured for different values of 𝑛. The highest accuracy was obtained when 𝑛 = 4.

Evaluation Criteria

Both automatic and human evaluations were carried out to access responses generated by various methods.

Automatic Evaluation

In automatic evaluation, the quality of the generated responses was evaluated from three perspectives: relevance, diversity, and style. The relevance was measured by BLEU [27] and ROUGE [28]. Specifically, the similarity between a generated response and a ground-truth response was evaluated using BLEU-1, BLEU-2, ROUGE-1, ROUGE-2, and ROUGE-L. The diversity was measured by Distinct-1 (Dist-1) and Distinct-2 (Dist-2), following the experiment by Li et al. [18]. The style was evaluated using "Style Correlation" (StyCor). The StyCor metric is defined as the correlation between the probability of the casual style 𝑝(𝑆=casual|𝑌 ) and the ground-truth level of the intimacy 3 . This correlation is high when both the predicted probability of the casual style and the intimacy level are high, or both are low (i.e., the probability of the polite style is high and the intimacy is low). It evaluates the extent to which the dialog model can control the style so that it generates a response in the casual (or polite) style when the user's level of intimacy is high (or low).

Human Evaluation

The quality of the generated responses was evaluated by human subjects. A hundred instances of responses were randomly chosen from the test set in the JID corpus. For each instance, responses were generated using the methods described in subsection 4.2 against the dialog context 𝑋. The responses were then evaluated by the subjects according to the following three criteria:

• Style Control: Does the response align with the appropriate style for the relationship between the two speakers? Annotators are also instructed to read the dialog context and guess the relationship between the speakers. • Relevance: Is the content of the response relevant and consistent with the context? • Fluency: Is the response natural, fluent, and free of grammatical errors?

The quality of responses was evaluated by assigning a score of 3 (appropriate), 2 (neutral), or 1 (inappropriate) for each of the three perspectives. Ten native Japanese speakers participated in the human evaluation. The inter-annotator agreement was measured using Fleiss's kappa [29].

Results

Results of Automatic Evaluation

Table 1 shows the results of the automatic evaluation. The bold indicates the best system for each criterion. The StyCor of our proposed method using ground-truth intimacy labels, I-S-GPT 𝑔𝑜𝑙𝑑 , was 0.366. It significantly outperformed the other baseline methods. Especially, the StyCor of I-S-GPT 𝑔𝑜𝑙𝑑 was much better than that of the rule-based method, Rule 𝑔𝑜𝑙𝑑 , which naively altered the polite and casual style generation by heuristics. These results indicated that our proposed method was superior at generating stylized responses based on the user's level of intimacy. When the user's intimacy was estimated, however, the StyCor of I-S-GPT 𝑎𝑢𝑡𝑜 was 0.103, which was better than STYLEDGPT but worse than DialoGPT. The poor StyCor of I-S-GPT 𝑎𝑢𝑡𝑜 may be due to the low accuracy (69%) of the intimacy estimation model. It was also supported by the large difference between I-S-GPT 𝑎𝑢𝑡𝑜 and I-S-GPT 𝑔𝑜𝑙𝑑 . Our proposed method is highly dependent on the performance of the intimacy estimation model. As for the relevance, S-GPT 𝑐𝑎 achieved the best BLEU, while DialogGPT achieved the best ROUGE. Our methods I-S-GPT 𝑎𝑢𝑡𝑜 and I-S-GPT 𝑔𝑜𝑙𝑑 were slightly worse for BLEU and obviously worse for ROUGE than the best system, but comparable to other baselines. As for the diversity, no significant difference of Dist-1 and Dist-2 was observed between the methods. From these results, it was found that the outstanding ability of the pre-trained dialog model (DialoGPT) to produce relevant and diverse responses was not remarkably damaged by incorporating the techniques of style control. Besides, no significant difference was found in relevance and diversity between I-S-GPT 𝑎𝑢𝑡𝑜 and I-S-GPT 𝑔𝑜𝑙𝑑 .

Results of Human Evaluation

The automatic evaluation revealed that the StyCor scores of the methods that automatically estimated the level of intimacy (I-S-GPT 𝑎𝑢𝑡𝑜 and Rule 𝑎𝑢𝑡𝑜 ) were insufficiently high. These two methods were excluded from the human evaluation process to reduce the burden on the annotators.

Table 2 shows the results of the human evaluation. The "Score" column indicates the average of scores assigned by the ten annotators. The "𝜅" column represents Fleiss's 𝜅, which indicates the agreement of scores between annotators. We also used Welch's test to verify whether there was a significant difference in scores between I-S-GPT 𝑔𝑜𝑙𝑑 and other methods. The "𝑝" column shows the 𝑝-value associated with this statistical test.

Style Control

The proposed method, I-S-GPT 𝑔𝑜𝑙𝑑 , achieved the highest score for style control. The 𝑝-values indicated that I-S-GPT 𝑔𝑜𝑙𝑑 was significantly better than the other methods, except for Rule 𝑔𝑜𝑙𝑑 . These results demonstrated that our proposed method was capable of generating responses in a more appropriate style. Rule 𝑔𝑜𝑙𝑑 was the second-best method, and both I-S-GPT 𝑔𝑜𝑙𝑑 and Rule 𝑔𝑜𝑙𝑑 were designed to control the style according to the level of intimacy. This confirms the validity of our approach to consider the user's level of intimacy to use polite and casual styles appropriately. However, the 𝜅 for style control was 0.13, indicating that the inter-annotator agreement was relatively low.

Relevance Although I-S-GPT 𝑔𝑜𝑙𝑑 was worse than the other methods in the automatic evaluation of relevance (as shown in Table 2), it achieved the highest score for relevance in the human evaluation. Nevertheless, no significant difference was observed. At least, the ability of the proposed method to generate responses relevant to the dialog context was comparable to that of the other baselines.

Fluency

As with the relevance score, the average score for fluency was the highest for the proposed method. However, a significant difference was only found between DialoGPT and I-S-GPT 𝑔𝑜𝑙𝑑 . The 𝜅 for fluency was higher than that for style control and relevance, indicating that the annotators exhibited greater consistency in evaluating the fluency of the responses. Computational Time Table 3 shows a comparison of the average time required for response generation per utterance across all test samples. A server with NVIDIA RTX A6000 48GB is used for the time measurements. DialoGPT exhibited the shortest generation time, followed by S-GPT, I-S-GPT, and Rule. S-GPT takes more time than DialoGPT due to the additional sampling and ranking strategy. In addition, I-S-GPT and Rule are slower than S-GPT because they require additional processing for the intimacy estimation.

Conclusion

This paper proposed a novel method of controlling the speech style of a dialog system according to the user's level of intimacy with the dialog system. Based on the PLM, which was a good dialog model, two loss functions were proposed to fine-tune it to generate responses in an appropriate style. In addition, the special token indicating the user's level of intimacy was added to the input of the dialog model. The results of automatic and human evaluations demonstrated that our proposed method outperformed the baseline for style control, indicating that the method could generate responses in a polite style when intimacy was low and a casual style when intimacy was high.

In the experiments, the accuracy of the intimacy estimation model was low, which caused a considerable decrease in the performance of the dialog model that used this intimacy estimation model. In the future, by improving the intimacy estimation model, we will enhance the style control ability of the dialog system under conditions where the ground-truth intimacy labels are not used.

It is our position that the study will not give rise to any significant ethical concerns. Our approach only controls speech styles according to the internal state of a user, and it does not introduce or exacerbate any ethical or social bias in a dialog system.