Semi-Supervised Models via Data Augmentation for Classifying Interactive A ective Responses

Abstract

We present semi-supervised models with data augmentation (SMDA), a semi-supervised text classi cation system to classify interactive a ective responses. SMDA utilizes recent transformer-based models to encode each sentence and employs back translation techniques to paraphrase given sentences as augmented data. For labeled sentences, we performed data augmentations to uniform the label distributions and computed supervised loss during training process. For unlabeled sentences, we explored self-training by regarding low-entropy predictions over unlabeled sentences as pseudo labels, assuming high-con dence predictions as labeled data for training. We further introduced consistency regularization as unsupervised loss after data augmentations on unlabeled data, based on the assumption that the model should predict similar class distributions with original unlabeled sentences as input and augmented sentences as input. Via a set of experiments, we demonstrated that our system outperformed baseline models in terms of F1-score and accuracy.

Keywords

• Semi-Supervised Learning
• Data Augmentation
• Deep Learn- ing
• Social Support
• Self-disclosure

-

A ect refers to emotion, sentiment, mood, and attitudes including subjective evaluations, opinions, and speculations [ 23 ]. Psychological models of a ect have been utilized by other extensive computational research to operationalize and measure users' opinions, intentions, and expressions. Understanding a ective responses with in conversations is an important rst step for studying a ect and has attracted a growing amount of research attention recently [ 20, 4, 19 ]. The a ective understanding of conversations focuses on the problem of how speakers use emotions to react to a situation and to each other, which can help better understand human behaviors and build better human-computerinteraction systems.

However, modeling a ective responses within conversations is relatively challenging since it is hard to quantify the a ectiveness [ 16 ] and there are no largescale labeled dataset about a ective levels in responses. In order to facilitate research in modeling interactive a ective responses, [ 8 ] introduced a conversation dataset, O MyChest, building from Reddit, and proposed two tasks: (1) Semi-supervised learning task: predict labels for Disclosure and Supportiveness in sentences based on a small amount of labeled and large unlabeled training data; (2) Unsupervised task: design new characterizations and insights to model conversation dynamics. The current work focused on the rst task.

With limited labeled data and large amount of unlabeled data being given, to alleviate the dependence on labeled data, we combine recent advances in language modeling, semi-supervised learning on text and data augmentations on text to form Semi-Supervised Models via Data Augmentation (SMDA). SMDA consists of two parts: supervised learning over labeled data (Section 4.1) and unsupervised learning over unlabeled data (Section 4.2). Both parts utilize data augmentations to enhance the learning procedures. Our contributions in this work can be summarized into three parts: analysed the O MyChest dataset in Section 3, proposed a semi-supervised text classi cation system to classify interactive a ective responses classi cation in Section 4 and described the experimental details and results in Section 5. 2

Related Work

Transformer-based Models : With transformer-based pre-trained models becoming more and more widely-used, pre-training and ne-tuning framework [ 7 ] with large pre-trained language models are applied into a lot of NLP applications and achieved state-of-the-art performances [ 18 ]. Language models [ 15, 7, 21 ] or masked language models [ 3, 10 ] are pre-trained over a large amount of text from Wikipedia and then ne-tuned on speci c tasks like text classi cations. Here we built our SMDA system based on such framework.

Data Augmentation on Text : When the amount of labeled data is limited, one common technique for handling the shortage of data is to augment given data and generate more training \augmented" data. Previous work has utilized simple operations like synonym replacement, random insertion, random swap and random deletion for text data augmentation [ 17 ]. Another line of research applied neural models for augmenting text by generating paraphrases via back translations [ 18 ] and monotone submodular function maximization [ 9 ]. Building on those prior work, we utilized back translations as our augment methods on both labeled and unlabeled sentences.

Semi-Supervised Learning on Text Classi cation : One alternative to deal with the lack of labeled data is to utilize unlabeled data in the learning process, which is denoted as Semi-Supervised Learning (SSL), since unlabeled data is usually easier to get compared to labeled data. Researchers has made use of variational auto encoders (VAEs) [ 2, 22, 6 ], self-training [ 11, 5, 12 ], consistency regularization [ 14, 13, 18 ] to introduce extra loss functions over unlabeled data to help the learning of labeled samples. VAEs utilize latent variables to reconstruct input labeled and unlabeled sentences and predict sentence labels with these latent variables; self-training adds unlabeled data with high-con dence predictions as pseudo labeled data during training process and consistency regularization forces model to output consistent predictions after adding adversarial noise or performing data augmentations to input data. We combined self-training, entropy minimization and consistency regularization in our system for unlabeled sentences. 3

Data Analysis and Pre-processing

Researching how human initiate and hold conversations has attracted increasing attention those days, as it can help us better understand how human behave over conversations and build better AI systems like social chatbot to communicate with people. In this section, we took a closer look at the conversation dataset, O MyChest [ 8 ], for better understanding and modeling interactive a ective responses. Speci cally, we describe certain characteristics of this dataset and our pre-processing steps. 3.1

Label De nition

For each comment of a post on Reddit, [ 8 ] annotated them with 6 labels: Information disclosure representing some degree of personal information in comments; Emotional disclosure representing comments containing certain positive or negative emotions; Support referring to comments o ering social support like advice; General support representing that comments are o ering general support through quotes and catch phrases, with Information support o ering speci c information like practical advice, and Emotional support o ering sympathy, caring or encouragement. Each comment can belong to multiple categories. In O MyChest corpus, there are 12,860 labeled sentences and over 420k unlabeled sentences for training, 5,000 unlabeled sentences for test. The label distributions of labeled sentences are showed in Fig. 1. To train and evaluate our

Original Augmented I'm crying a lot of tears of joy Right now I'm crying a lot of right now. happy tears.

Stepdad will be the one walking It will be my stepfather walking me down the aisle when I get me down the aisle when I married. get married.

Hope you have a nice day. I hope you have a good day.

Your best e ort, both of you bBeostthsohfoty.ou are giving it your Plan your transition back to Plan your move back to a job working outside of the home. outside your own home.

I am so freaking happy for you! I'm so excited for you! systems, we randomly split the given labeled sentence set into train, development and test set. The data statics are shown in Table 1. We tuned hyper-parameters and chose best models based on performance on dev set, and reported model's performance on test set. We utilized XLNet-cased-based tokenizer3 to split each sentence into tokens. We showed the cumulative sentence length distribution in Fig. 2, 95% comments have less than 64 tokens. Thus we set the maximum sentence length to 64, and remained the rst 64 tokens for sentences that exceed the limit. As for data augmentations, we made use of back translation with German as middle language to generate paraphrases for given sentences. Speci cally, we loaded translation model from Fairseq4, translated given sentences from English to German, and then translated them back to English. Also to increase the diversity of generated paraphrases, we employed random sampling with a tunable temperature (0.8) instead of beam search for the generation. We describe some examples in Table 2. 4

Method

We convert this 6-class a ective response classi cation task into 6 binary classi cation tasks, namely whether each sentence belongs to each category or not 3 https://huggingface.co/transformers/model_doc/xlnet.html#xlnettokenizer 4 https://github.com/pytorch/fairseq (labeled with 1 or 0). For each binary classi cation task, given a set of labeled sentences consisting of n samples S = fs1; :::; sng with labels L = fl1; :::; lng, where li 2 f0; 1g2, and a set of unlabeled sentences Su = fs1u; :::; sumg, our goal is to learn the classi er f (^ljs; i); i 2 [1; 6]. Our SMDA model contains several components: Supervised Learning (Section 4.1) for labeled sentences, Unsupervised Learning (Section 4.2) for unlabeled sentences, and Semi-Supervised Objective Function (Section 4.3) to combine labeled and unlabeled sentences.

Generating Balanced Labeled Training Set As shown in Fig. 1, the dis

tribution is very unbalanced with respect to General support, Info support and Emo support. In order to get more training sentences with these three types of support and make these three binary classi cation sub-tasks learn-able with a more balanced training set, we performed data augmentations over sentences with these three labels. Speci cally, we paraphrased each sentence by 4 times via back translations and regarded that the augmented sentences have the same labels as original sentences. The comparison distributions are shown in Fig. 3

Supervised Learning for Labeled Sentences For each input labeled sen

tence si, we used XLNet [ 21 ] g(:) to encode it into hidden representation hi = g(si), and then passed them though a 2-layer MLP to predict the class distribution l^i = f (hi). Since these sentences have speci c labels, we optimize the cross entropy loss as supervised loss term:

LS (si; li) =

X li log f (g(si)) (1)

(a) Before Augmentation (b) After Augmentation Paraphrasing Unlabeled Sentences We rst performed back translations once for each unlabeled sentence siu 2 Su to generate the augmented sentence su;a; :::; sum;ag in the same manner we described before. set Su;a = f 1 Guessing Labels for Unlabeled Sentences For an unlabeled sentence siu, we utilized g(:) and f (:) in Section 4.1 to predict the class distribution: To avoid the prediction being so close to uniform distribution, we generate lowentropy guessing labels ~liu by a sharpening function [ 1 ]: ^liu = f (g(siu)) ~liu =

(^liu) T1 jj(^liu) T1 jj1 (2) (3) where jj:jj1 is l1-norm of the vector. When T ! 0, the guessed label becomes an one-hot vector.

Self-training for Original Sentences Inspired by self-training where model is also trained over unlabeled data with high-con dence predictions as their labels, in SMDA, with our guessed labels ~liu with respect to original unlabeled sentence siu, we added such pair (siu; ~liu) into training by minimize the KL Divergence between them:

Ls(siu) = KL(f (g(siu))jj~liu)

Entropy Minimization for Original Sentences One common assumption in

many semi-supervised learning methods is that a classi er's decision boundary should not pass through high-density regions of the marginal data distribution [ 5 ]. Thus for original unlabeled sentence siu, we added another loss term to minimize the entropy of model's output:

Le(siu) =

X f (g(siu)) log f (g(siu))

Consistency Regularization for Augmented Sentences With the assump

tion that the model should predict similar distributions with input sentences before and after augmentations, we minimized the KL Divergence between outputs with original sentence siu as input and augmented sentence siu;a as input:

Lc(siu) = KL(^liujjf (g(siu;a))) Combining all the loss terms for unlabeled sentences, we de ned our unsupervised loss terms as:

LU (siu) = Ls(siu) + Le(siu) + Lc(siu) 4.3

Semi-Supervised Objective Function

We combined the supervised and unsupervised learning described above to form our overall semi-supervised objective function:

L = E(si;li)2(S;L)LS (si; li) +

Esiu2Su LU (siu) where

is the balanced weight between supervised and unsupervised loss term. 5 5.1

Experiments

Model Setup

In SMDA 5, we only used single model for each task without jointly training and parameter sharing. That is, we trained six separate classi ers on these tasks. 5 The codes and data split will be released later. (4) (5) (6) (7) (8)

Inspired by recent success in pre-trained language models, we utilized the pretrained weights of XLNet and followed the same ne-tuning procedure as XLNet. We set the initial learning rate for XLNet encoder as 1e-5 and other linear layers as 1e-3. The batch size was selected in f32; 64; 128; 256g. The maximum number of epochs is set as 20. Hyper-parameters were selected using the performance on development set. The sharpen temperature T was selected in f0:3; 0:5; 0:8g depending on di erent tasks. The balanced weight between supervised learning loss and unsupervised learning loss term started from a small number and grew through training process to 1. 5.2

Results

Our experimental results are shown in Table 3. We compared our proposed SMDA with BERT and XLNet in terms of accuracy(%) and Macro F1 score. BERT and XLNet achieved similar performance since they both obey the pretraining and ne-tuning manner. When combining with augmented and more balanced labeled data, massive unlabeled data, our SMDA achieved best performance across six binary-classi cation tasks. And we submitted the classi cation results on given unlabeled test set. In this work, we focused on identifying disclosure and supportiveness in conversation responses based on a small labeled and large unlabeled training data via our proposed semi-supervised text classi cation system : Semi-Supervised Models via Data Augmentation (SMDA). SMDA utilized supervised learning over labeled data and conducted self-training, entropy minimization and consistency regularization over unlabeled data. Experimental results demonstrated that our system outperformed baseline models signi cantly.

References

1. Berthelot , D. , Carlini , N. , Goodfellow , I.J. , Papernot , N. , Oliver , A. , Raffel , C. : Mixmatch: A holistic approach to semi-supervised learning . CoRR abs/1905 .02249 ( 2019 )

2. Chen , M. , Tang , Q. , Livescu , K. , Gimpel , K. : Variational sequential labelers for semi-supervised learning . In: Proc. of EMNLP ( 2018 )

3. Devlin , J. , Chang , M.W. , Lee , K. , Toutanova , K. : BERT: Pre-training of deep bidirectional transformers for language understanding . In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , Volume 1 (Long and Short Papers). pp. 4171 { 4186 . Association for Computational Linguistics, Minneapolis, Minnesota (Jun 2019 )

4. Ernala , S.K. , Rizvi , A.F. , Birnbaum , M.L. , Kane , J.M. , De Choudhury , M. : Linguistic markers indicating therapeutic outcomes of social media disclosures of schizophrenia . Proceedings of the ACM on Human-Computer Interaction 1(CSCW) , 43 ( 2017 )

5. Grandvalet , Y. , Bengio , Y. : Semi-supervised learning by entropy minimization . In: Proceedings of the 17th International Conference on Neural Information Processing Systems . pp. 529 { 536 . NIPS'04, MIT Press, Cambridge, MA, USA ( 2004 )

6. Gururangan , S. , Dang , T. , Card , D. , Smith , N.A. : Variational pretraining for semisupervised text classi cation . CoRR abs/ 1906 .02242 ( 2019 )

7. Howard , J. , Ruder , S. : Universal language model ne-tuning for text classi cation . In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . pp. 328 { 339 . Association for Computational Linguistics, Melbourne, Australia (Jul 2018 )

8. Jaidka , K. , Singh , I. , Jiahui , L. , Chhaya , N. , Ungar , L. : A report of the CL-A O MyChest Shared Task at A ective Content Workshop @ AAAI. In: Proceedings of the 3rd Workshop on A ective Content Analysis @ AAAI (A Con2020) . New York, New York ( February 2020 )

9. Kumar , A. , Bhattamishra , S. , Bhandari , M. , Talukdar , P. : Submodular optimization-based diverse paraphrasing and its e ectiveness in data augmentation . In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , Volume 1 (Long and Short Papers). pp. 3609 { 3619 . Association for Computational Linguistics, Minneapolis, Minnesota (Jun 2019 ). https://doi.org/10.18653/v1/ N19 - 1363, https://www.aclweb.org/anthology/N19-1363

10. Lample , G. , Conneau , A. : Cross-lingual language model pretraining . CoRR abs/ 1901 .07291 ( 2019 )

11. Lee , D.H. : Pseudo-label : The simple and e cient semi-supervised learning method for deep neural networks . ICML 2013 Workshop : Challenges in Representation Learning (WREPL) ( 07 2013 )

12. Meng , Y. , Shen , J. , Zhang, C., Han , J .: Weakly-supervised neural text classi cation . In: Proceedings of the 27th ACM International Conference on Information and Knowledge Management . pp. 983 { 992 . CIKM '18, ACM , New York, NY, USA ( 2018 ). https://doi.org/10.1145/3269206.3271737, http://doi.acm. org/10 .1145/ 3269206.3271737

13. Miyato , T. , Dai , A.M. , Goodfellow , I. : Adversarial training methods for semisupervised text classi cation . In: International Conference on Learning Representations ( 2017 )

14. Miyato , T. , Maeda , S. , Koyama , M. , Ishii , S. : Virtual adversarial training: A regularization method for supervised and semi-supervised learning . IEEE Trans. Pattern Anal. Mach. Intell . 41 ( 8 ), 1979 { 1993 ( 2019 )

15. Peters , M. , Neumann , M. , Iyyer , M. , Gardner , M. , Clark , C. , Lee , K. , Zettlemoyer , L. : Deep contextualized word representations . In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , Volume 1 (Long Papers). pp. 2227 { 2237 . Association for Computational Linguistics, New Orleans, Louisiana (Jun 2018 )

16. Warriner , A.B. , Shore , D.I. , Schmidt , L.A. , Imbault , C.L. , Kuperman , V. : Sliding into happiness: A new tool for measuring a ective responses to words . Canadian Journal of Experimental Psychology/Revue canadienne de psychologie experimentale 71(1) , 71 ( 2017 ). https://doi.org/10.1037/cep0000112, https://app.dimensions.ai/details/publication/pub.1084126691andhttp: //europepmc.org/articles/pmc5334777?pdf=render

17. Wei , J.W. , Zou , K. : EDA: easy data augmentation techniques for boosting performance on text classi cation tasks . CoRR abs/ 1901 .11196 ( 2019 )

18. Xie , Q. , Dai , Z. , Hovy , E. , Luong , M.T. , Le , Q.V. : Unsupervised data augmentation for consistency training . arXiv preprint arXiv: 1904 . 12848 ( 2019 )

19. Yang , D. , Kraut , R.E. , Smith , T. , May eld, E., Jurafsky , D. : Seekers, providers, welcomers, and storytellers: Modeling social roles in online health communities . In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems . p. 344 . ACM ( 2019 )

20. Yang , D. , Yao , Z. , Seering , J. , Kraut , R.: The channel matters: Self-disclosure, reciprocity and social support in online cancer support groups . In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems . p. 31 . ACM ( 2019 )

21. Yang , Z. , Dai , Z. , Yang , Y. , Carbonell, J.G., Salakhutdinov , R. , Le , Q.V. : Xlnet: Generalized autoregressive pretraining for language understanding . CoRR abs/ 1906 .08237 ( 2019 )

22. Yang , Z. , Hu , Z. , Salakhutdinov , R. , Berg-Kirkpatrick , T. : Improved variational autoencoders for text modeling using dilated convolutions . CoRR abs/1702 .08139 ( 2017 )

23. Zhang , P.: The a ective response model: A theoretical framework of a ective concepts and their relationships in the ict context . Management Information Systems Quarterly (MISQ) 37 , 247 { 274 (03 2013 ). https://doi.org/10.25300/MISQ/ 2013 /37.1. 11