Multimodal Emotion Recognition in the Political Domain in Spanish Ronghao Pan1 1 Facultad de Informática, Universidad de Murcia, Campus de Espinardo, 30100, Spain Abstract Recognizing and understanding emotions is essential for improving human-computer interaction and has diverse applications, including supporting mental health and increasing customer satisfaction in various industries. As interactive systems are increasingly expected to perceive, understand, and express emotions like humans, automatic emotion recognition becomes crucial in practical contexts. Social networks, which are the main channels for rapid information dissemination, often display users’ emotions through their shared messages, which has a significant impact on the spread of misinformation. By integrating emotion recognition tools into disinformation detection, we can improve our ability to counter information manipulation and foster a more transparent, evidence-based information environment. This research aims to develop, evaluate, and apply multimodal emotion recognition methods, particularly in political contexts, to combat misinformation and analyze the use of fallacies in political discourse. Keywords Emotion Analysis, Multimodal Analysis, Natural Language Processing 1. Introduction and background Emotions are complex responses, both psychological and physiological, that people experience to various stimuli. These reactions are often associated with specific thoughts, feelings, and behaviors [1]. Emotions can be positive, such as joy and love, or negative, such as sadness and fear. They are essential for making decisions, interacting socially, and adapting to different situations. Understanding and recognizing emotions is critical not only for improving human-computer interaction, but also for a wide range of applications, from supporting mental health to improving customer satisfaction in a variety of industries [2]. Therefore, automatic Emotion Recognition (ER) plays an important role in real-world appli- cations, as interactive machines are increasingly expected to be able to perceive, understand, and express emotions on the same level as humans [3]. In this context, Emotion Recognition (ER) involves the identification of emotional cues from different media, such as text, facial expressions, voice tones, and body language. ER systems that integrate two or more features of different types, such as text and audio, are called multimodal emotion recognition. Social networks have become the primary platform for the rapid dissemination of information to the general public. Despite its benefits, the lack of effective regulatory measures has led to a Doctoral Symposium on Natural Language Processing, 26 September 2024, Valladolid, Spain. Envelope-Open ronghao.pan@um.es (R. Pan) Orcid 0009-0008-7317-7145 (R. Pan) © 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR ceur-ws.org Workshop ISSN 1613-0073 Proceedings flood of fake news and rumors on the Internet, making it difficult to distinguish between truthful information and misinformation, thus affecting the efficiency of information exchange on social networks [4]. For this reason, they face the critical challenge of spreading misinformation. Differentiating between authentic and misleading content is a significant challenge, leading many users to unknowingly share information they believe to be authentic. Users’ emotions, conveyed through their shared messages, play a significant role in the spread of disinformation [5]. The use of emotions for disinformation detection is an area that has already been explored and experimented with, and emotions have been shown to be complementary to disinforma- tion detection [6]. Integrating emotion recognition tools and techniques into disinformation detection will strengthen the ability to combat information manipulation and promote a more transparent and evidence-based information environment. However, challenges remain, such as the lack of public datasets in Spanish and problems related to multimodality. Regarding fallacies, a strong relationship with emotions is observed, but studies exploring this possibility to improve detection are still lacking. Fallacies are errors in logical reasoning that lead to invalid or misleading arguments and are common in political discourse, advertising, and everyday conversation. The objective of this PhD thesis is to design and evaluate of different multimodal ER ap- proaches and apply them in different domains such as political to identify and address misin- formation, as well as to understand the use of fallacies in political discourse. We focused our proposal on Spanish because it is the third most used language on the Internet, only after English and Chinese. Despite its importance, there remains a noticeable lack of linguistic resources to perform multimodal ER in some domains, such as the political domain. 2. Research Hypotheses The research hypotheses in this paper are related to the multimodal emotion detection approach for solving misinformation and fallacy identification tasks in the political domain. Our first research hypothesis is that the multimodal ER approach improves the performance of text-based emotion analysis. Our second research hypothesis is that the multimodal ER approach improves misinformation and fallacy identification in the political domain. To validate these research hypotheses, we established the following objectives: • OB01. Creation of a multimodal corpus in Spanish for emotion recognition and for disin- formation detection. This corpus will be built from a crawler that extracts videos, texts, and audios from different YouTube channels related to politics, sports and entertainment, as well as from recordings of sessions of the Congress of Deputies. • OB02. Evaluation of different multimodal approaches that combine features of different modalities for ER. In this case, it can be the combination of audio with text, text with images or the combination of all three. Moreover, we are also evaluating our proposal by participating in several shared task regarding multimodal classification from different workshops. • OB03. Identification of misinformation and fallacies using ER. To validate our hypothesis, we are compiling different multimodal political disinformation corpus and making them available to the scientific community. 3. Methodology and Experiments In this section, we describe the methodology and experiments we have created to help us validate our hypotheses. 3.1. Multimodal Emotion Recognition Our research is primarily focused on addressing the limitations of current approaches to detect- ing misinformation and deception in the political domain through the analysis of emotions. In the preliminary stages of our research, we recognized the need for emotion analysis in a wide range of applications, from supporting mental health to improving customer satisfaction in a variety of industries. In addition, we identified the current lack of multimodal resources in Spanish for ER in speech, despite the fact that Spanish is one of the most widely spoken languages in the world, with millions of speakers in different continents. The few available datasets suffer in some cases from poor annotation, making it difficult to teach models to accurately identify emotions. In other cases, voice recordings are made by professionals under conditions that do not reflect real-life situations, resulting in a lack of authenticity in emotional expression [7] and poor results in real-life scenarios [8]. Therefore, one of the current challenges in Spanish speech ER is the creation of more robust and representative datasets. To address this problem, we have created and published a new corpus for multimodal ER in Spanish (Spanish MEACorpus 2023) [9], which contains 13.16 hours of speech divided into 5,129 labeled segments, taking into account Ekman’s six basic emotions (disgust, anger, happiness, sadness, fear, neutral) and annotated by three members of our research group (two men and one woman, all middle-aged). In this paper, we have also evaluated different multimodal approaches that combine speech representation techniques and linguistic models to perform emotion classification. We have evaluated approaches ranging from simple text-based emotion detection to approaches based on the fusion of automatic speech recognition models such as Wav2Vec2- BERT [10] with pre-trained models such as BETO. Among the model fusion approaches, we have evaluated the late fusion approach, which consists of concatenating or averaging the outputs obtained in Wav2Vec-BERT and BETO; fusion with multi-head cross-attention mechanism, which incorporates a cross-attention mechanism in the hidden state vector of the last layer to better capture complex audio-text relationships, generate richer contextual representations, and increase the generalization capability of the model; and ensemble learning, which is based on combining feature sets using ensemble learning, where the output of each neural network model (Wav2Vec2-BERT and MarIA) is combined by averaging the predictions of each emotion (mean) or selecting the prediction with the highest probability (maximum). As a contribution to my Ph.D. thesis, this paper shows that multimodal ER models combining audio and text perform better than unimodal models based on text alone, demonstrating that audio and text features complement each other. In addition, the late fusion approach with concatenation strategy is identified to perform better. Moreover, this dataset has been used as a basis for the organization of the EmoSPeech task [11] in IberLEF 2024, which consists of two subtasks: text-based automatic ER and multimodal automatic ER. The novelty of this task lies in its multimodal approach to ER, analyzing the performance of language models on Spanish MEACorpus 2023. Furthermore, we have participated in several shared task related to multimodal classification in several relevant evaluation forums such as CLEF and SemEval. First, in SemEval-2024 Task 4[12], which objective is identify persuasive techniques in memes. For this task, we evaluated LlaVa to extract image descriptions and combine them with the meme text. Our system performed well in all subtasks, achieving the tenth best result with a Hierarchical F1 of 64.774%, the fourth best in Subtask 2a with a Hierarchical F1 of 69.003%, and the eighth best in Subtask 2b with a Macro F1 of 78.660%. Second, in SemEval-2024 Task 10 [13], focused on recognizing and reasoning about emotional changes in conversations. This task included different languages such as English and Hindi. Our best result was the 6th position in Subtask 2 with an F1 score of 26%. Third, on EXIST 2024 [14], focused on the identification and categorization of sexism in memes. We used the CLIP model to extract the embedded text and image, and then combined them by diagonal multiplication to obtain the classification models. We ranked 33rd in sexism identification and 18th in both source intent and sexism categorization. Participating in these evaluation forums has allowed me to validate the proposed approaches for multimodal classification that combine features from images and text. In the future, the plan is to integrate representations from different modalities such as text, images, and audio to improve the task of emotion identification. 3.2. Disinformation and Fallacy Detection Using Emotion Analysis Recent advances in Artificial Intelligence (AI) and the emergence of Large Language Models (LLMs) like Instruct-GPT [15], ChatGPT, and GPT-4 [16], have made it easier to generate false information that appears very convincing [17]. Therefore, there is an urgent need to detect misinformation efficiently and effectively. Most studies provide high-level summaries of the methods, techniques, and features used to detect rumors and fake news, while others discuss possible applications of these methods, including posture detection and source detection [6]. Currently, disinformation detection approaches consist of three main components: (i) the dataset used to develop them, (ii) the methods used to perform the detection, and (iii) the features considered in these methods [6]. Most datasets are obtained from social media platforms such as Twitter, Facebook, and Sina Weibo, or from fact-checking websites such as Snopes1 , Factcheck2 and PolitiFact3 . Detection methods can be divided into those based on conventional Machine Learning (ML) and Deep Learning (DL) techniques. We tested new approaches to misinformation detection, such as using the “5W1H” technique commonly used by journalists to clearly and explicitly present the most important information in a news item and to assess the reliability of the language presented in the news. We participated in FLARES at IberLEF 2024 [18], which uses the 5W1H technique to assess the reliability of language in news items. This task is divided into two subtasks: (i) 5W1H item identification and (ii) 5W1H-based reliability assessment. We participated in both tasks. For Task 1, we developed a Named Entity Recognition (NER) model using tight transformation models such as BERT and MarIA, and integrating Part-Of-Speech (POS) features and Syntactic Dependencies (Dep). Our BETO + POS + Dep model achieved the 1 https://www.snopes.com/ 2 https://www.factcheck.org/ 3 https://www.politifact.com/ second best result with a score of 56.778%. In Task 2, which focused on assessing the reliability of 5W1H entities, our approach based on contextual adaptation of the MarIA model achieved the best result with a score of 65.820%. Within the affective feature for disinformation detection, dual emotion features seek to account for the importance of considering different emotional perspective in disinformation identification, i.e., both the emotion of the publisher, which refers to emotions conveyed in an original post that starts a thread on social networks, and the social emotion, which refers to emotions expressed in follow-up posts that respond and/or comment on the original post [6]. Recent research, such as [5] and [19], has demonstrated the importance of ER in the fight against misinformation and fallacies in political and media discourse. In [5], several methods were developed that use ER to detect misinformation in social networks. Thus, ER is complementary to misinformation detection because it provides insight into emotional manipulation, content intentionality, argument quality, and public response. Integrating ER tools and techniques into misinformation detection will strengthen the ability to combat information manipulation and promote a more transparent and evidence-based information environment. Emotion-based misinformation detection combines emotion and sentiment information with other features to maximize performance. According to [6], these methods are classified into: (i) Methods combining emotion with other text-based features; (ii) Mining of dual emotions that consider emotions in news posts and reactions on social media; (iii) Methods based on tree or graph structures that utilize relationships between social media posts to model information diffusion; (iv) Methods based on temporal information, that consider propagation patterns and changes in reader emotions; (v) Multitask learning that focuses on multiple tasks simultaneously and leveraging shared information; and (vi) Multimodal methods that consider images or audio attached to textual posts. Recent studies combine text and image features to improve misinformation and rumor detection [20] [21]. According to the study conducted by [6] on emotion-based methods for misinformation detection, it was observed that the majority of misinformation datasets, whether public or private, are in English. One of the current challenges in emotion-based misinformation detection using large language models is multimodality. Continued advances in technology have led to an increasing trend towards multimodality, as it is indeed common for people to supplement textual content in posts with images or videos, while on platforms such as YouTube or TikTok, video has become the dominant medium for sharing information. As a result, it is becoming increasingly important to explore methods that can address the challenges of multimodality and adapt to the ever-changing characteristics of social media communication. Fallacious arguments were originally defined as defective inferences, i.e., types of arguments that are logically invalid. More recently, from a pragmatic perspective, they have been defined as violations of the typical performance rules of a particular ideal type of argumentative engagement [22], and as inappropriate shifts between different types of dialog, noting that the intended move is inappropriate within the applied pragmatic context [23]. Although it occurs in a variety of settings, political debate serves as a natural testing ground for this form of fallacious reasoning. For example, the ad hominem fallacy, which attacks the person or entity making the argument rather than the argument itself, is one of the most commonly cited fallacies in political debate. Such arguments can sound convincing and are intended to mislead the audience into believing the validity of the argument [24]. Currently, there are several types of fallacies, such as hasty generalization, ad hominem, ad populum, false causality, circular reasoning, appeal to emotions, red herring, false deduction, credibility fallacy, false dilemma, straw man, and intention [25]. After analyzing these fallacies, we found that some of them are closely related to the emotion of contempt. For example, contempt may lead one to disqualify the opponent by attacking his character, intelligence, or morality instead of addressing his arguments, which leads to ad hominem fallacies. In addition, contempt can motivate the presentation of arguments in an exaggerated or oversimplified manner in order to distort the opponent’s argument, which is a straw man fallacy. Another fallacy related to the emotion of contempt is the appeal to emotion, which consists of appealing to emotions rather than presenting a logical argument, i.e., using disgust or contempt to evoke a strong emotional response that distracts from the logic of the argument. For example, the question “Do you really want to support someone who advocates such disgusting practices?” for the opponent. In conclusion, the use of emotions for disinformation detection is an area that has already been explored and experimented with, and emotions have been shown to be complementary to disinformation detection. Integrating emotion recognition tools and techniques into disinfor- mation detection will strengthen the ability to combat information manipulation and promote a more transparent and evidence-based information environment. However, challenges remain, such as the lack of public datasets in Spanish and problems related to multimodality. Regarding fallacies, a strong relationship with emotions is observed, but studies exploring this possibility to improve detection are still lacking. Therefore, my thesis will focus on the creation of a multimodal public dataset in Spanish for emotion identification and misinformation detection, aiming to address the lack of public datasets in these two research areas. In addition, different multimodal approaches for emotion- based misinformation detection, which is one of the main challenges raised in [6], will be explored. As for fallacies, which are errors in logic or reasoning that can lead to incorrect conclusions, emotions can be a key feature in their identification. Therefore, it is also proposed to integrate emotion recognition into fallacy detection using a public dataset. We are currently in the phase of identifying emotions related to misinformation and fallacies, as well as collecting more multimodal data in the Spanish MEACorpus 2023. In addition, we are adding examples of the emotion of contempt to the corpus in order to improve the performance of the models in identifying misinformation and fallacies. 4. Conclusions and Further work This Ph.D. thesis aims to investigate multimodal emotion recognition and apply it in the political domain to detect misinformation or fallacies. To achieve this goal, first, the current state of the art in the field was reviewed. Second, a multimodal dataset of audio and text, called Spanish MEACorpus 2023 [9], has been compiled and published for emotion recognition in Spanish and different multimodal approaches have been evaluated, from text-based ER to approaches based on the fusion of automatic speech recognition models such as Wav2Vec2-BERT with pre- trained models such as BETO. Thirdly, we have participated in different shared tasks of different evaluation forums such as CLEF, IberLEF, and SemEval to validate our proposed approaches for the task of multimodal classification, emotion detection and disinformation detection. We found that emotion improves the identification of mental illness [26]. Besides, we participated in several tasks using emotion recognition to improve mental illness identification and hopeful discourse. We are currently in the process of identifying which emotions are most correlated with misinformation and fallacies. We have found that the emotion of contempt is strictly related to certain fallacies, such as ad hominem, straw man, and appeal to emotion. To this end, we are extending the Spanish MEACorpus 2023 dataset by adding examples of the emotion contempt in order to create a consistent model for detecting misinformation and fallacies. Acknowledgments This work is part of the research projects LaTe4PoliticES (PID2022-138099OB-I00) funded by MICIU/AEI/10.13039/501100011033 and the European Regional Development Fund (ERDF)-a way of making Europe and LT-SWM (TED2021-131167B-I00) funded by MICIU/AEI/10.13039/ 501100011033 and by the European Union NextGenerationEU/PRTR, and ”Services based on language technologies for political microtargeting“ (22252/PDC/23) funded by the Autonomous Community of the Region of Murcia through the Regional Support Program for the Transfer and Valorization of Knowledge and Scientific Entrepreneurship of the Seneca Foundation, Science and Technology Agency of the Region of Murcia. Mr. Ronghao Pan is supported by the Programa Investigo grant, funded by the Region of Murcia, the Spanish Ministry of Labour and Social Economy and the European Union - NextGenerationEU under the “Plan de Recuperación, Transformación y Resiliencia (PRTR)”. References [1] P. Ekman, et al., Basic emotions, Handbook of cognition and emotion 98 (1999) 16. [2] N. Ahmed, Z. A. Aghbari, S. Girija, A systematic survey on multimodal emotion recognition using learning algorithms, Intelligent Systems with Applications 17 (2023) 200171. URL: https://www.sciencedirect.com/science/article/pii/S2667305322001089. doi:https://doi. org/10.1016/j.iswa.2022.200171 . [3] S.-W. Byun, J.-H. Kim, S.-P. Lee, Multi-modal emotion recognition using speech features and text-embedding, Applied Sciences 11 (2021) 7967. [4] D. A. Scheufele, N. M. Krause, Science audiences, misinformation, and fake news, Proceedings of the National Academy of Sciences 116 (2019) 7662–7669. URL: https://www.pnas.org/doi/abs/10.1073/pnas.1805871115. doi:10.1073/pnas.1805871115 . arXiv:https://www.pnas.org/doi/pdf/10.1073/pnas.1805871115 . [5] I. V., S. M. Thampi, Misinformation detection in social networks using emotion analysis and user behavior analysis, Pattern Recognition Letters 182 (2024) 60–66. URL: https: //www.sciencedirect.com/science/article/pii/S0167865524001089. doi:https://doi.org/ 10.1016/j.patrec.2024.04.007 . [6] Z. Liu, T. Zhang, K. Yang, P. Thompson, Z. Yu, S. Ananiadou, Emotion detection for misinformation: A review, Information Fusion 107 (2024) 102300. URL: https: //www.sciencedirect.com/science/article/pii/S1566253524000782. doi:https://doi.org/ 10.1016/j.inffus.2024.102300 . [7] M. B. Akçay, K. Oğuz, Speech emotion recognition: Emotional models, databases, features, preprocessing methods, supporting modalities, and classifiers, Speech Communication 116 (2020) 56–76. URL: https://www.sciencedirect.com/science/article/pii/S0167639319302262. doi:https://doi.org/10.1016/j.specom.2019.12.001 . [8] Y. Singh, S. Goel, An efficient algorithm for recognition of emotions from speaker and language independent speech using deep learning, Multimedia Tools and Applications 80 (2021). doi:10.1007/s11042- 020- 10399- 2 . [9] R. Pan, J. A. García-Díaz, M. Ángel Rodríguez-García, R. Valencia-García, Spanish meacorpus 2023: A multimodal speech–text corpus for emotion analysis in spanish from natural environments, Computer Standards & Interfaces 90 (2024) 103856. URL: https://www.sciencedirect.com/science/article/pii/S0920548924000254. doi:https://doi. org/10.1016/j.csi.2024.103856 . [10] L. Barrault, Y.-A. Chung, M. C. Meglioli, D. Dale, N. Dong, M. Duppenthaler, P.-A. Duquenne, B. Ellis, H. Elsahar, J. Haaheim, et al., Seamless: Multilingual expressive and streaming speech translation, arXiv preprint arXiv:2312.05187 (2023). [11] R. Pan, J. A. García-Díaz, M. Á. Rodríguez-García, F. García-Sanchez, R. Valencia-García, Overview of EmoSPeech 2024@IberLEF: Multimodal Speech-text Emotion Recognition in Spanish, Procesamiento del Lenguaje Natural 73 (2024). [12] D. Dimitrov, F. Alam, M. Hasanain, A. Hasnat, F. Silvestri, P. Nakov, G. Da San Martino, Semeval-2024 task 4: Multilingual detection of persuasion techniques in memes, in: Proceedings of the 18th International Workshop on Semantic Evaluation, SemEval 2024, Mexico City, Mexico, 2024. [13] S. Kumar, M. S. Akhtar, E. Cambria, T. Chakraborty, Semeval 2024 – task 10: Emotion discovery and reasoning its flip in conversation (ediref), in: Proceedings of the 2024 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Association for Computational Linguistics, 2024. URL: https://arxiv.org/abs/ 2402.18944. [14] L. Plaza, J. Carrillo-de Albornoz, V. Ruiz, A. Maeso, B. Chulvi, P. Rosso, E. Amigó, J. Gonzalo, R. Morante, D. Spina, Overview of exist 2024 – learning with disagreement for sexism identification and characterization in social networks and memes (extended overview, in: Working Notes of CLEF 2024- Conference and Labs of the Evaluation Forum, Springer, 2024. [15] L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al., Training language models to follow instructions with human feedback, Advances in neural information processing systems 35 (2022) 27730–27744. [16] R. OpenAI, Gpt-4 technical report. arxiv 2303.08774, View in Article 2 (2023). [17] Y. Yan, B. Li, J. Feng, Y. Du, Z. Lu, M. Huang, Y. Li, Research on the impact of trends related to chatgpt, Procedia Computer Science 221 (2023) 1284–1291. URL: https://www.sciencedirect.com/science/article/pii/S187705092300875X. doi:https://doi. org/10.1016/j.procs.2023.08.117 , tenth International Conference on Information Technology and Quantitative Management (ITQM 2023). [18] R. Sepúlveda-Torres, A. Bonet-Jover, I. Diab, I. Guillén-Pacho, I. Cabrera-de Castro, C. Badenes-Olmedo, E. Saquete, M. T. Martín-Valdivia, P. Martínez-Barco, L. A. Ureña- López, Overview of flares at iberlef 2024: Fine-grained language-based reliability detection in spanish news, Procesamiento del Lenguaje Natural 73 (2024). [19] E. Mancini, F. Ruggeri, P. Torroni, Multimodal fallacy classification in political debates, in: Y. Graham, M. Purver (Eds.), Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers), Association for Computational Linguistics, St. Julian’s, Malta, 2024, pp. 170–178. URL: https://aclanthology. org/2024.eacl-short.16. [20] R. Kumari, N. Ashok, P. K. Agrawal, T. Ghosal, A. Ekbal, Identifying multimodal misinformation leveraging novelty detection and emotion recognition, J. Intell. Inf. Syst. 61 (2023) 673–694. URL: https://doi.org/10.1007/s10844-023-00789-x. doi:10.1007/ s10844- 023- 00789- x . [21] Z. Jin, J. Cao, H. Guo, Y. Zhang, J. Luo, Multimodal fusion with recurrent neural networks for rumor detection on microblogs, Proceedings of the 25th ACM international conference on Multimedia (2017). URL: https://api.semanticscholar.org/CorpusID:22504814. [22] F. H. van Eemeren, Crucial concepts in argumentation theory, Amsterdam University Press, 2001. [23] F. H. V. Eemeren, R. Grootendorst, Fallacies in pragma-dialectical perspective, Argumen- tation 1 (1987) 283–301. doi:10.1007/bf00136779 . [24] P. Goffredo, M. Chaves, S. Villata, E. Cabrio, Argument-based detection and classification of fallacies in political debates, in: H. Bouamor, J. Pino, K. Bali (Eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, Singapore, 2023, pp. 11101–11112. URL: https://aclanthology. org/2023.emnlp-main.684. doi:10.18653/v1/2023.emnlp- main.684 . [25] F. L. C. y José A. Troyano y Fernando Enríquez y F. Javier Ortega, Detección y clasificación de falacias prototípicas y espontáneas en español, Procesamiento del Lenguaje Natural 71 (2023) 53–62. URL: http://journal.sepln.org/sepln/ojs/ojs/index.php/pln/article/view/6542. [26] A. Salmerón-Ríos, J. A. García-Díaz, R. Pan, R. Valencia-García, Fine grain emotion analysis in spanish using linguistic features and transformers, PeerJ Computer Science 10 (2024) e1992. doi:10.7717/peerj- cs.1992 .