Maintaining consistent personas is essential for dialogue models. However, dialogue models can generate lfuent but inconsistent responses with persona. We observed that some inconsistent responses often contain similar but inconsistent words. In this poster, we propose a method that uses unlikelihood loss to separate semantics of similar but inconsistent words. To get such words, we leverage Word2Vec to construct the negative word set. And ConceptNet is used to remove consistent noise words from negative word set and add antonyms. Experiments demonstrate that our method improves the persona consistency of dialogue generation.
With the success of existing dialogue models on generating human-like responses, dialogue
models are required to express their own personality. Zhang et al. [
Unlikelihood training is a technique developed for removal of repetition in language model
completions. Li et al. [
In this poster, we construct negative word set to separate semantics of similar but inconsistent words. Firstly, we obtain coarse negative word set by Word2Vec. Secondly, we use ConceptNet [4] to remove synonyms and add antonyms. Thirdly, we use the unlikelihood loss to compute the loss of negative word set, which assigns a low probability to inconsistent words. The experiments show that our method can generate more consistent responses.
2. i am a man.
2.2.1. Likelihood Loss Our task is to train a generative model to generate a persona consistent response. Formally, given a set of persona texts = { 1, 2, … , }, and an query , to generate a response which should consistent with persona. Here , and are sentences, which are consist of some words, such as = { 1, 2, … , }, where is the length of .
Likelihood training is commonly used in text generation models. Some pre-trained generative models trained with likelihood can generate fluent and meaningful responses. For a sample { , , } , the likelihood uses maximum likelihood estimation (MLE) to compute loss: = − log ( ( ∣ , )
|| ) = − ∑ log ( ( ∣ , , =1 < )) (1) where is current word needed to be predicted, < are previous words before and ( ∣ , , < ) represents the probability of predicted by model conditional on , , < . 2.2.2. Unlikelihood Loss Likelihood training increases the probability of true word and decreases the probability of all other words. On the contrary, unlikelihood training decreases the probability of negative words. The unlikelihood (UL) loss can be defined as:
= − log (1 − ( ∣ , ) where is the negative word set of current word .
|| ) = − ∑ ∑ log (1 − ( ∣ , , =1 ∈ < )) (2)
2.3.1. Negative Word Set MLE leverages the previous context of the word to predict the current word, which results in words with similar contexts having similar semantics. Therefore, the model might generate similar but inconsistent words. One solution is separating the semantics of similar words, which can be done by UL training. The core of UL training is to construct negative word set. Unlike Welleck et al. [5], our negative word set contains inconsistent words with current word conditional on persona. As shown in fig. 1, the generated word “20” is inconsistent with “26” in persona. The negative word set of “26” maybe contains “20”, “25” and “30” et al. 2.3.2. Word2Vec The challenge is how to construct negative word set used for UL loss. Word2Vec, a method of learning word embedding, follow the distributional Hypothesis: words that occur in the same contexts tend to have similar meanings. Word2Vec leverages the previous and following context to predict current word, which is similar to MLE. We learning word embedding of all words on dataset by Word2Vec. we approximatively regard similar words computed by cosine similarity of word embedding as negative word set. For example, the negative word set of “man” computed by Word2Vec contains “male”, “boy”, and “girl” et al. 2.3.3. ConceptNet We also observed that there are some noise words in the negative word set constructed by Word2Vec. The synonym, hyponym, and hypernym have similar context but are consistent with the word, so we should remove them from the coarse negative word set. Fortunately, ConceptNet [4], a knowledge graph containing common sense knowledge, provides these three relations of one word. For example, “male” is a synonym of “man” and “dog” is a hyponym for “pet”. Besides, ConceptNet also provides antonyms that can be added to negative word set. For example, “man” and “woman” is a pair of antonym, and they have similar context but opposite semantics.
We use GPT2 [
We verify our method on PersonaChat [
Table 1 shows that the model trained by UL loss achieves better results on all metrics than the base model. Higher consistent ratio and lower contradictory ratio indicate our method can separate semantics of similar but inconsistent words. The consistency of responses is further improved after using ConceptNet, which means the knowledge such as synonyms provided by ConceptNet is useful to construct higher-quality negative word set.
In this poster, we propose a method to construct negative word set for unlikelihood training to separate semantics of similar but inconsistent words. Experiments demonstrate that our method can improve persona consistency of dialogue generation. In future work, we are interested in leveraging more eficient loss and constructing more appropriate data to improve the persona consistency of dialogue generation. Annual Meeting of the Association for Computational Linguistics, Online, Association for Computational Linguistics, 2020, pp. 4715–4728. [4] R. Speer, J. Chin, C. Havasi, Conceptnet 5.5: An open multilingual graph of general knowledge, in: Proceedings of the 31st AAAI Conference on Artificial Intelligence, San Francisco, California, USA, AAAI Press, 2017, pp. 4444–4451. [5] S. Welleck, I. Kulikov, S. Roller, E. Dinan, K. Cho, J. Weston, Neural text generation with unlikelihood training, in: Proceedings of the 8th International Conference on Learning Representations, Addis Ababa, Ethiopia, OpenReview.net, 2020.