Emotions are an integral part of human communication, and accurately detecting and interpreting them from textual data holds significant potential for numerous applications. The Valence-Arousal-Dominance (VAD) dimensional model provides a rich framework for capturing the nuanced emotional states conveyed through text. This paper presents an in-depth exploration of text-based emotion detection using VAD analysis and machine learning techniques. In this paper, we propose a novel machine-learning model specifically designed to detect VAD dimensions from textual data. Through empirical evaluation and comparison with existing methods, we analyze the results of our proposed model in accurately identifying emotions expressed in text. This paper describes the ISTC-CNR participation in the EmotivITA task at EVALITA 2023, showcasing our team's eforts and findings in the context of the competition.
1. Introduction
ence, influencing our perception, behavior, and overall
well-being. Traditionally, emotions have been studied
using categorical models [
nEvelop-O (M. Mongiovì) (M. Mongiovì) 2https://sites.google.com/view/emotivita/the-tasks performance of the VAD MNLI-XML-ROBERTA-based system with the baseline BERT-based model provided by the organizers. Finally, Section 4 summarizes the key ifndings, discusses the practical implications, and outlines potential directions for future research.
grained emotions along the continuous VAD dimensions. Our approach encompasses both the dimensional (subtask A) and multi-dimensional (sub-task B) VAD regression tasks, following a unified methodology framework. To accomplish this, we introduce a supervised model developed within a constrained mode, leveraging the EmoITA. Section 2.1 provides a comprehensive overview of our methodology, ofering detailed insights into the key components and techniques employed in our NLIbased approach. Section 2.2 provide an overview of the prompts utilized during the training phase. sentence convey?). For arousal, the prompt is “quanto è eccitante la frase?” (how exciting is the sentence?). Lastly, for dominance, the prompt is “quanto è controllata l’emozione?” (how controlled is the emotion?). These prompts are crafted to succinctly encapsulate the essence of each emotional dimension using easily understood language. Their purpose is to guide the model’s training process toward discerning specific emotional dimensions accurately. In the multi-dimensional emotion analysis task, we utilize a single prompt to predict all emotion dimensions. The prompt used is “valence, arousal, dominance dell’emozione?” (valence, arousal, dominance of the emotion?). In this case, the prompt assists the model in comprehending the comprehensive framework of the task during the training phase, providing contextual information. By leveraging these prompts, our objective is to enhance the model’s capacity to capture and comprehend the intricate nuances of emotional dimensions in text.
2.1. VAD MNLI-XML-RoBERTa model
We present the VAD MNLI-XML-RoBERTa model, a fine- tiveness of our proposed approach and compare it with
tuned version of the multilingual XML-RoBERTa system, the baseline method provided by the competition’s
orwhich has been developed using an automatically trans- ganizers using EmoITA [
els, each focused on detecting the degree of valence, arousal, and dominance values individually. During training, each model focuses on a specific target dimension and is associated with a specific prompt (Sect. 2).
Tables 3.1 and 3.1 showcase the results obtained by evaluating our MNLI-based model on 2′063 items from 2.2. Prompting the Italian EmoBank test set provided by the organizers. For the dimensional emotion analysis task, we submitted We introduce a set of tailored prompts designed to pre- two diferent runs corresponding to distinct configuradict the values of valence, arousal, and dominance in tions of the fine-tuned VAD MNLI-XML-RoBERTa model. text. For the dimensional emotion analysis task, we de- During the model training phase, the first run incorpovelop three prompts, each targeting a specific emotion rates 99% of the data from the development set, while dimension. The prompt for valence is “quanta positiv- the second run utilizes the entire development set. Each ità esprime la frase?” (how much positivity does the emotional dimension is evaluated based on Pearson’s correlation coeficient (Table 3.1) and Mean Absolute Errors (Table 3.1). To assess the efectiveness of our model, we
4. Discussion compare its performance against a baseline BERT-based model provided by the organizers. Our NLI-based model demonstrates slightly higher Pearson’s correlation val- As anticipated, performance varied significantly across ues for valence compared to the baseline, while the other diferent emotion values in both dimensional (i.e. submetrics show slightly lower scores. Notably, valence ex- task A) and multi-dimensional (i.e. sub-task B) emotion hibits the highest performance in terms of correlation, regression tasks. The concept of valence demonstrated while arousal and dominance present more challenges, consistent interpretation, resulting in higher correlation with the baseline model achieving a correlation of only scores for all models in both analyses. Conversely, detect0.64%. ing concepts like “arousal” and “dominance” proved more challenging in the context of short text documents. Com3.2. Multi-dimensional Emotion Analysis paring the two models, the BERT-based model yielded the best results, while the NLI-based model exhibited slightly We present the performance of models designed for lower performance in both tasks. This discrepancy can be detecting valence, arousal, and dominance in a multi- attributed to the multilingual nature of the MNLI-XMLdimensional setting. In this setting, both the baseline and RoBERTa model, which grants it greater adaptability but MNLI-based models predict all three values by utilizing limits its ability to identify nuanced and ambiguous terms all three VAD dimensions during the training phase. For like “dominance” and “excitement” within text. Additionthis task, we submitted a single run where we employed ally, the NLI-based models employed a generic prompt for all the available development set data for training the every item in the dataset as a hypothesis. This approach model. We report the performances of the VAD MNLI- potentially underutilized the model’s capabilities as the XML-RoBERTa system for each emotion dimension, eval- prompt lacked the necessary explanatory power to efecuating them in terms of Pearson’s correlation coeficient tively tackle challenging regression tasks. In conclusion, (Table 3.2) and Mean Absolute Error (Table 3.2). We com- while the models demonstrated strengths in capturing pare our results with the baseline performance provided valence, improvements are needed to enhance their perby the organizers. formance in discerning arousal and dominance. This in
Both the baseline and NLI-based models demonstrate cludes refining the model’s adaptability to diverse linguissuperior performance in detecting valence, achieving a tic contexts and developing more informative prompts correlation of approximately 80%. As previously men- for comprehensive analysis of emotional dimensions. tioned, the scores for the arousal and dominance dimensions are comparatively lower than valence. Specifically, the highest correlation values observed for arousal and 5. Conclusion dominance values are 0.652 and 0.654, respectively.
multi-dimensional regression scenarios, leveraging a finetuned XML-RoBERTa classifier trained and evaluated on the EmoITA. We present the results of both the dimensional and multi-dimensional configurations’ models and compare them with the performance of the BERT-based model provided by the competition’s organizers. This evaluation provides valuable insights into the efectiveness of our proposed approach in capturing and analyzing emotional dimensions in textual data.