One of the crucial features in developing a dialog system is the choice of an appropriate speech style. This paper proposes a novel method for training a dialog model that can efectively control the style of a response. Specifically, the dialog model generates responses in a polite style when the user exhibits a low level of intimacy with the system and in a casual style when the user shows a high level of intimacy. Using a pre-trained language model (PLM) as a base dialog model, two loss functions are proposed for fine-tuning the PLM to generate responses in an appropriate style. One is the intimacy-aware word-level loss, which serves to ensure that the dialog model generates a polite or casual word when the user's level of intimacy is low or high. The other is the intimacy-aware sentence-level loss, which functions to increase the probability of the polite style of the generated utterance when the user's level of intimacy is low, and vice versa. The results of both automatic and human evaluations in the experiments demonstrate that the proposed method is more efective than the baselines in generating responses that align with the user's degree of intimacy. Furthermore, the proposed method exhibits comparable relevance and fluency to the PLM, indicating that the losses for the style control do not diminish the PLM's exceptional capacity for generating relevant and fluent responses.
Dialog systems that freely chat with users on a wide range of topics have attracted a great deal of
attention in recent years [
The goal of this research is to develop a dialog system that flexibly controls speech styles according to the user. Specifically, concerning the user’s intimacy with the dialog system, a response is generated in a polite style when the user’s level of the intimacy is low, and in a casual style when the level of the intimacy is high. To achieve this, we propose a method to incorporate knowledge necessary for style control by fine-tuning a dialog model based on a pre-trained language model (PLM) that is capable of generating a variety of responses consistent with the dialog context. A new loss function for fine-tuning a dialog model is designed so that the model generates polite or casual responses when the level of the intimacy is low or high, where the level of the intimacy is estimated from the user’s past utterances.
The contributions of this paper are summarized as follows: • We develop a dialog system that estimates the user’s level of the intimacy and controls the polite and casual styles in generating responses accordingly. • We propose an approach to incorporate knowledge for style control into an existing outstanding
PLM-based dialog model. • We demonstrate the efectiveness of the proposed method by both automatic and manual evaluations.
Several methods have been developed for the generation of responses in a specific style. Niu and Bansal
defined the task of generating responses in a predefined style, such as polite or rude [
Based on the aforementioned studies on maintaining a style in response generation, more recent
methods have been developed to add the capability of style control to a well-developed existing dialog
model. Sun et al. trained a dialog model using reinforcement learning, in which responses similar to
the ground-truth response and including style-related tokens got a higher reward [
Although the above previous studies can generate natural stylized responses, they are limited in their ability to handle a single style. In contrast, our method enables the control of multiple styles according to the user’s mental state.
Several studies focused on the emotional state of the user during a dialog. Skowron et al. showed
that interactive expression of emotions in response to the user’s feelings can significantly contribute to
enhancing the enjoyment of the chat and the emotional connection between the user and the system
[
The proposed method learns a dialog model that can adapt its style to either polite or casual by the user’s level of intimacy, which is automatically estimated from the user’s historical utterances. Figure 1 shows an overview of the proposed method. Let us suppose that a dialog model generates a response to a user for a given dialog context = {1, 1, · · · , , }, where a system () and a user ( ) make an utterance alternately. Figure 1 exemplifies the case of n = 4. The intimacy estimation model employs the user’s previous utterances = {1, · · · , } as input and determines whether the user’s level of intimacy with the dialog system is high or low. The dialog model accepts the context and the estimated intimacy level as input and generates the response +1 in a casual style when the user’s intimacy is high and in a polite style when the user’s intimacy is low.
To learn the above dialog model, we extend STYLEDGPT [
STYLEDGPT employs DialoGPT [
Word-Level Loss First, a style language model ( ) is trained in advance. Using a style corpus, , consisting of only texts in a given style, GPT-2 is trained as an autoencoder, i.e., the same sentence ∈ is given as input and output for fine-tuning.
Let ( |) be a dialog model that returns a response for a given dialog context . The loss is computed for each dialog sample (, ) in the training data . is a sequence of words denoted by = {1, · · · , }. Let pY = {1 , · · · , } be the distribution of the predicted probability of the next word given by the dialog model ( |). Also, let p^Y = {^1 , · · · , ^ } be the distribution of the probability of predicting the next word given by the style language model ( ) when the output of the dialog model is taken as input of the style language model. The distance between pY and p^Y is defined as word-level loss as in Eq. (1).
= (pY||p^Y) d=ef ∑︁ ( ||^ ) =1 (1) is the Kullback-Leibler (KL) divergence of the two probability distributions. This loss causes pY to approach p^Y, i.e., the dialog model is trained to produce utterances in the specified style. Sentence-Level Loss First, a style discrimination model (| ) is trained in advance. It identifies whether a sentence is written in a given style . This model is trained on a dataset that consists of , a corpus of sequences written in the specific style, as positive samples, and , a general dialog corpus, as negative samples.
The loss is computed for each dialog sample (, ) ∈ . Let ^ be a response generated by the dialog model ( |) for the input , and (|^ ) be the probability that the style of ^ is coincident with the style . Then, the sentence-level loss is defined as in Eq. (2).
= − log (|^ ) This loss causes the dialog model ( |) to produce utterances in the style .
Negative Log-likelihood Loss The two losses mentioned above are designed to take into account the style of a response. Fine-tuning a model with only these losses may result in a lack of consistency between a context and a generated response. Therefore, the negative log-likelihood loss (Eq. (3)) is also used, which is a common loss for training a dialog model. ( |) is the probability that the dialog model generates a ground-truth response from , where (, ) is a sample in .
= − ( |)
We modify the word-level and sentence-level losses in STYLEDGPT to control the style of the response according to the user’s level of intimacy.
First, an intimacy estimation model (|) is trained. This model predicts , the user’s level of intimacy with a dialog system, given the user’s past utterances () as input. In our model, is defined as either low or high. The intimacy estimation model is pre-trained using a dialog corpus annotated with the speaker’s intimacy.
To handle both polite and casual styles in response generation, two style corpora are prepared. One is , which consists of polite-style sentences, and the other is , which consists of casual-style sentences.
Intimacy-aware Word-Level Loss First, the language models of the polite and casual styles, ( ) and ( ), are pre-trained using the corpora and , respectively. Next, the word-level loss of the polite style, , is computed as in Eq. (1). It evaluates how likely a response is to be polite. Similarly, the word-level loss of the casual style, , is calculated. Finally, the intimacy-aware word-level loss, , is defined as the weighted sum of these two losses (Eq. (4)). (=low|) and (=high|) are the weights for and , which are the probability that the user’s level of intimacy is low and high, respectively. (2) (3) (4) = (=low|) · + (=high|) · This loss is expected to encourage the generation of more polite tokens when intimacy is low and more casual tokens when intimacy is high.
Intimacy-aware Sentence-Level Loss First, we train a style discrimination model ′(| ) that identifies a style of a sentence , where is either polite or casual. The style discrimination model is trained in advance on training data where utterances in are samples of the polite class and those in are samples of the casual class.
Let ^ be the output of the dialog model ( |) for a given context . Then, the style of ^ is identified by the style discrimination model and (=polite|^ ) and (=casual|^ ) are obtained. Following the sentence-level loss of STYLEDGPT, the intimacy-aware sentence-level loss, , is defined as the weighted sum of the logarithms of these probabilities, using the two probabilities (=low|) and (=high|) as weights (Eq. (5)).
= −( =low|) · log (=polite| ) − (=high|) · log (=casual| ) (5) This loss is expected to learn the dialog model to generate utterances in the polite style when the intimacy is low and in the casual style when the intimacy is high.
Training Objective Eq. (6) shows the total loss, which is a weighted sum of the two losses concerning a style ( and ) and a general response loss ().
= · + · + · (6) , , and are hyperparameters representing the weight of each loss.
In addition to incorporating the information of user’s intimacy into the loss functions, the user’s level of intimacy is explicitly given in an input to the dialog model. Specifically, the level of intimacy is identified by (|), and then the intimacy label is added to the input as follows: • When =low : <l> <s> context </s> • When =high : <h> <s> context </s> <l> and <h> are special tokens indicating the low and high intimacy classes, respectively. <s> and </s> are special tokens indicating the beginning and end of the dialog context. This additional input allows the dialog model to generate responses in an appropriate style that matches the identified level of intimacy.
To enhance the ability of the dialog model to generate appropriately styled utterances, the
samplingand-rank decoding strategy [
The style score Score() is a weighted sum of the style probabilities of , as in Eq. (7). The weights are the probabilities of the low and high intimacy predicted by the history of the user’s utterances . A greater style score indicates that a response is generated in the polite (or casual) style and the user’s level of intimacy is low (or high).
Score() = (=low|) · (=polite|) + (=high|) · (=casual|) (7)
The content score Score() is defined as the probability that the dialog model ( |) outputs the response candidate when the dialog context is an input, as shown in Eq. (8). This score evaluates the relevance of to .
Score() = (|)
The final score Score() is defined as Eq. (9). The hyperparameter determines the relative weighting of the two scores.
Score() = (1 − ) · Score() + · Score() (8) (9)
Dialog Corpus with Intimacy Level Our in-house dialog corpus annotated with intimacy labels was used to evaluate the proposed method. This corpus consists of recorded and transcribed dialogs of approximately ten minutes, conducted between two speakers. For each dialog, the intimacy labels of each of the two speakers to his/her dialog partner are annotated on a five-point scale. The statistics of the corpus are as follows: the number of subjects who participated in the conversations is 19, the number of conversations is 54, and the total number of utterances is 6,984. Hereafter, we refer to this corpus as the “Japanese Intimacy Dialog Corpus” or “JID corpus” for short.
The 54 dialogs in the JID corpus were divided into three subsets: a training set of 33 dialogs, a validation set of 9, and a test set of 12. As mentioned in Section 3, the dialog model accepts the preceding dialog context of the user and the system, = {1, 1, · · · , , }, as input and generates the subsequent response +1 as output. Hereafter, the pair of a dialog context and its corresponding response, denoted by (, +1), will be referred to as an instance of response. One speaker in the corpus was designated as the system and the other as the user to extract a dialog context and response. The first × 2 utterances and the next utterance in a dialog were extracted as (, +1). This procedure was then repeated, with the utterance shifted one by one, to obtain multiple instances of responses. Finally, 4,032, 921, and 1,284 instances of responses were obtained as the training, validation, and test data, respectively.
We also used this corpus to train an intimacy estimation model. Let = {1, · · · , } be the user’s utterance extracted from the dialog context in an instance of response, and let be the intimacy label for the dialog. The intimacy label was designated as “low” when the corresponding score in the JID corpus was 1 or 2, or “high” when the value was 3, 4, or 5. The intimacy estimation model, (|), is a binary classification model that takes as input and estimates the intimacy label . The model was trained using samples (, ) in the training and validation data and its performance was evaluated using the test data.
Style Corpus Two style corpora are required to train style language models and a style discrimination
model: and . The KeiCO corpus [
To train the polite and the casual style language model, ( ) and ( ), all utterances in and , respectively, were utilized. To train the style discrimination model ′(| ), a total of 23,248 utterances were used, comprising 9,957 utterances in and 13,301 utterances in . The remaining 100 utterances (50 utterances each) were used to evaluate the style discrimination model.
The following methods, including our proposed methods, were compared in the experiment.
• DialoGPT [
If the performance of the intimacy estimation model is inadequate, misclassification of the level of intimacy may prevent the learning of the stylized dialog model. To verify the efectiveness of our approach to control the style of response in terms of intimacy, I-S-GPT was also evaluated. It can be regarded as an ideal system that always correctly estimate the user’s intimacy. In this method, in Eq. (4) and (5), the probability of the level of intimacy was approximated by the five-point intimacy score () in the JID corpus as (=low|) ≃ 1 − 5 and (=high|) ≃ 5 . The additional input described in subsection 3.3 was also given by the ground-truth intimacy score, that is, <l> is added when is between 1–2, while <h> is added when is between 3–5.
A method using a Large Language Model (LLM) for style-controlled generation can be considered as a baseline. However, when a prompt is provided to ChatGPT to guess the user’s level of intimacy and respond in an appropriate style, the generated responses are almost always polite. Therefore, prompting-based LLM is not included in this experiment.
Style Language Model and Discrimination Model The style language models ( ) and ( )
were obtained by fine-tuning GPT-2. The architecture of the style language model consists of an
embedding layer, a transformer module, and a decoding layer of GPT-2. The pre-trained model was
japanese-gpt2-medium1, which had been trained on a large-scale Japanese dialog dataset. The learning
rate was set to 5e−4 , the batch size to 4, and the epoch to 20. The Adam optimizer [
The style discrimination model ′(| ) was also obtained by fine-tuning the GPT-2 model. The
architecture of the style discrimination model consists of an embedding layer, a transformer module,
and a classification layer of GPT-2. The same pre-trained model used to train the style language model
was fine-tuned using the Adam optimizer with the same hyperparameters. The style discrimination
model was evaluated using the 100 utterances not used for training. Its accuracy was 64%.
Intimacy Estimation Model Bidirectional Encoder Representations from Transformers (BERT) [
The low accuracy indicates that intimacy estimation is a dificult task. Our error analysis shows that there are few indicative words that are highly related to the speaker’s intimacy. For example, in the sentiment analysis task, “pleasant” and “happy” are indicative words for positive emotions, and “sad” and “unhappy” are ones for negative emotions. However, such indicative words are rare in the intimacy estimation task. Another possible reason for the poor performance is the lack of training data. One of the possible directions is to apply semi-supervised learning to compensate for small amounts of labeled data with large amounts of unlabeled data.
Dialog Model The dialog model described in subsection 4.2 was obtained by fine-tuning GPT-2. The same pre-trained model1 used for training the style language models was utilized for fine-tuning the dialog model. As for the hyperparameters, the learning rate was set to 1e−18 , the batch size to 1, and the epoch to 10. The Adam optimizer was used to fine-tune the model.
The parameters , , and in Eq. (6) were set to 0.45, 0.45, and 0.1, respectively. These values
were optimized on the validation data according to the StyCor criterion, which will be described in §4.4.
As for the sampling-and-rank decoding strategy, the hyperparameters were set to the same values as
those used in STYLEDGPT [
The length of a dialog context = {1, 1, · · · , , } was set to 8, i.e., the parameter was set to 4. In the preliminary experiment to evaluate the intimacy estimation model, the accuracy of the model was measured for diferent values of . The highest accuracy was obtained when = 4.
Both automatic and human evaluations were carried out to access responses generated by various methods.
Automatic Evaluation In automatic evaluation, the quality of the generated responses was evaluated
from three perspectives: relevance, diversity, and style. The relevance was measured by BLEU [
Human Evaluation The quality of the generated responses was evaluated by human subjects. A hundred instances of responses were randomly chosen from the test set in the JID corpus. For each instance, responses were generated using the methods described in subsection 4.2 against the dialog context . The responses were then evaluated by the subjects according to the following three criteria: • Style Control: Does the response align with the appropriate style for the relationship between the two speakers? Annotators are also instructed to read the dialog context and guess the relationship between the speakers. • Relevance: Is the content of the response relevant and consistent with the context? • Fluency: Is the response natural, fluent, and free of grammatical errors?
The quality of responses was evaluated by assigning a score of 3 (appropriate), 2 (neutral), or 1
(inappropriate) for each of the three perspectives. Ten native Japanese speakers participated in the
human evaluation. The inter-annotator agreement was measured using Fleiss’s kappa [
Table 1 shows the results of the automatic evaluation. The bold indicates the best system for each criterion. The StyCor of our proposed method using ground-truth intimacy labels, I-S-GPT, was 0.366. It significantly outperformed the other baseline methods. Especially, the StyCor of I-S-GPT was much better than that of the rule-based method, Rule, which naively altered the polite and 3The five-scale score is normalized to values between 0 and 1. estimation model. It was also supported by the large diference between I-S-GPT and I-S-GPT. Our proposed method is highly dependent on the performance of the intimacy estimation model.
As for the relevance, S-GPT achieved the best BLEU, while DialogGPT achieved the best ROUGE. Our methods I-S-GPT and I-S-GPT were slightly worse for BLEU and obviously worse for ROUGE than the best system, but comparable to other baselines. As for the diversity, no significant diference of Dist-1 and Dist-2 was observed between the methods. From these results, it was found that the outstanding ability of the pre-trained dialog model (DialoGPT) to produce relevant and diverse responses was not remarkably damaged by incorporating the techniques of style control. Besides, no significant diference was found in relevance and diversity between I-S-GPT and I-S-GPT.
The automatic evaluation revealed that the StyCor scores of the methods that automatically estimated the level of intimacy (I-S-GPT and Rule) were insuficiently high. These two methods were excluded from the human evaluation process to reduce the burden on the annotators. assigned by the ten annotators. The “ ” column represents Fleiss’s , which indicates the agreement of scores between annotators. We also used Welch’s test to verify whether there was a significant diference in scores between I-S-GPT and other methods. The “” column shows the -value associated with this statistical test.
Style Control The proposed method, I-S-GPT, achieved the highest score for style control. The -values indicated that I-S-GPT was significantly better than the other methods, except for Rule . These results demonstrated that our proposed method was capable of generating responses in a more appropriate style. Rule was the second-best method, and both I-S-GPT and Rule were designed to control the style according to the level of intimacy. This confirms the validity of our approach to consider the user’s level of intimacy to use polite and casual styles appropriately. However, the for style control was 0.13, indicating that the inter-annotator agreement was relatively low. Relevance Although I-S-GPT was worse than the other methods in the automatic evaluation of relevance (as shown in Table 2), it achieved the highest score for relevance in the human evaluation. Nevertheless, no significant diference was observed. At least, the ability of the proposed method to generate responses relevant to the dialog context was comparable to that of the other baselines. Fluency As with the relevance score, the average score for fluency was the highest for the proposed method. However, a significant diference was only found between DialoGPT and I-S-GPT . The for fluency was higher than that for style control and relevance, indicating that the annotators exhibited greater consistency in evaluating the fluency of the responses. Computational Time Table 3 shows a comparison of the average time required for response generation per utterance across all test samples. A server with NVIDIA RTX A6000 48GB is used for the time measurements. DialoGPT exhibited the shortest generation time, followed by S-GPT, I-S-GPT, and Rule. S-GPT takes more time than DialoGPT due to the additional sampling and ranking strategy. In addition, I-S-GPT and Rule are slower than S-GPT because they require additional processing for the intimacy estimation.
This paper proposed a novel method of controlling the speech style of a dialog system according to the user’s level of intimacy with the dialog system. Based on the PLM, which was a good dialog model, two loss functions were proposed to fine-tune it to generate responses in an appropriate style. In addition, the special token indicating the user’s level of intimacy was added to the input of the dialog model. The results of automatic and human evaluations demonstrated that our proposed method outperformed the baseline for style control, indicating that the method could generate responses in a polite style when intimacy was low and a casual style when intimacy was high.
In the experiments, the accuracy of the intimacy estimation model was low, which caused a considerable decrease in the performance of the dialog model that used this intimacy estimation model. In the future, by improving the intimacy estimation model, we will enhance the style control ability of the dialog system under conditions where the ground-truth intimacy labels are not used.
It is our position that the study will not give rise to any significant ethical concerns. Our approach only controls speech styles according to the internal state of a user, and it does not introduce or exacerbate any ethical or social bias in a dialog system.