Integrating Subjectivity Analysis for Disinformation Detection in Spanish

Abstract

The widespread availability of internet access has revolutionized information consumption and creation, as evidenced by high online engagement in Spain. However, this digital scene also promotes the propagation of disinformation driven by ideological and economic motivations. This research undertakes a comprehensive analysis of subjectivity-related linguistic cues to identify textual features that can diferentiate deceptive content. Adopting an integrative perspective, this study address disinformation, which is misleading information designed, presented and promoted to intentionally cause public harm or for profit. In this study we focus on analyzing its pragmatic dimension, manifested in linguistic choices conveying author's imprint. This has been explored within psychological research though the study of deception, but its specific linguistic expression in the creation of disinformation remains a underexplored area. A more profound understanding of this pragmatic component of disinformation promises to enhance detection models by capturing the intentional manipulation inherent in such content. We argue that the way information is presented is more critical than the content itself in disinformation, contrasting with the traditional objective aspiration of news reporting. Our methodology employs a multidisciplinary approach, integrating analytical and empirical techniques to investigate this phenomenon. The anticipated impact of this research is to significantly improve the ability to recognize deceptiveness in written Spanish news through the analysis of subjectivity markers.

Keywords

• eol>NLP
• disinformation
• deception
• explainability
• subjectivity
• linguistic markers
• inclusive IA

-

and Zafarani [ 8 ] consider that disinformation supposes a greater threat and is harder to identify than unintentionally false information, as it seems as truth to mislead the public on purpose.

This way, our work sets of an integrative approach to the complex issue of disinformation, situating it within the broader landscape of false information research across diverse academic fields. Unlike diferent forms of false information, disinformation is characterized by a intentional and pragmatic dimension, reflected on linguistics elements that have been the focus of dedicated studies on deception. While the phenomenon of deception has often been explored from the psychological discipline, its manifestation in the linguistic elements and patterns used by the creator of disinformation remains an underexplored area. A deeper understanding on this line will enable the development and application of disinformation detection models with a more pragmatically informed perspective to capture the intentional manipulation embedded within disinformation.

Thus, the psycholinguistic component of disinformation is reflected in linguistic elements that are analyzed from a pragmatic perspective. At this point, the how of information publication is more important than the what. News conventionally aspire for objectivity in reporting events. However, the act of journalistic selection and structuring inherently involves filtering and leaves a linguistic imprint [ 9 ]. This filtering can range from minimal to substantial, with intentional manipulation for specific outcomes or profit potentially leading to disinformation. Building on this, our objective is to examine the communicative style of information to uncover authorial subjectivity cues that could be linked to the strategic filtering of information aimed at deceiving the public, focused on disinformation in Spanish.

2. Background and Related work

As previously stated, our research aims to analyze and detect disinformation through linguistic analysis of subjectivity markers. The existing background for this specific disinformation analysis through a linguistic lens is limited. Given this scarcity, the related work presented in this PhD thesis extends to include studies on linguistic analysis of false information in a broader sense and the phenomenon of deception itself. Thus, we will review diferent research topics related to the analysis of linguistic elements for the detection and analysis of features indicating deceptiveness, ranging from a broad analysis of disinformation in general to the review of more specific tasks such as the linguistic analysis of subjectivity, as presented in Figure 1.

Previous studies have tried to delimit the linguistic markers that allow the detection of the falsehood or veracity of a message. Early research by DePaulo et al. [ 10 ] identified 158 cues to deception through a controlled writing experiment, presenting a psychological foundation for linguistic analysis of falsehood. Hauch et al. [ 11 ]’s meta-analysis of 44 studies found up to 79 markers, highlighting the importance of certainty, emotion expression, distancing, details, and cognitive process expressions in deceptive texts. Zhou et al. [ 12 ] explored information richness (text, image, video) and specific word categories (persuasive, emotional, etc.) in misinformation. These studies provide a basis for a taxonomy of deception markers and inform the current state-of-the-art for fake news phenomena in English.

Research approaches on deception detection ofer valuable techniques for truth assessment in text, drawing from legal methodologies like Criteria-Based Content Analysis (CBCA) or Interpersonal Deception Theory (IDT), among others. In Zhou et al. [ 13 ] they identified linguistic cues, suggesting deceivers use longer but less complex messages with more negative emotions. However, they noted the potential relevance of various linguistic features (quantity, informality, etc.) in discrimination. This suggestion was supported by Almela et al. [ 14 ], who used a framework from Mihalcea et al. [ 15 ] and found 2nd and 3rd person pronouns to be strong indicators of untruthfulness, likely influenced by topic dependence and the emotional involvement of the sender. Similarly, in the thesis elaborated by Picornell [ 16 ] it was found a higher frequency of first-person singular pronouns ( I, Me, My) in deceptive statements.

Studying subjectivity is crucial for disinformation detection due to its broad coverage of author’s implication markers. Vieira et al. [ 17 ] hypothesizes that the degree of objectivity distinguishes news from disinformation, emphasizing the analysis of textual sequences containing both truthful and fabricated fragments. In this study they found greater subjectivity in disinformation. Similarly, Shokri et al. [18] showed Large Language Models (LLMs) can detect subjectivity based on linguistic features, derived from the classification proposed by [ 19] between news and opinionated texts. Ott et al. [20] improved deceptive opinion spam detection using psycholinguistic and n-gram features. This aligns with the linguistic profiling perspective proposed by Brunato et al. [ 21], which identifies sender “imprints” and is relevant to detecting deception through linguistic analysis.

Focusing on methodological approaches on NLP within machine learning (ML), Part-of-Speech (POS) tagging is widely employed, giving morphosyntactic tags to words based on their context [22]. For the specific task of automatically extracting linguistic markers, machine learning algorithms have proven its applicability. For instance, in Abonizio et al. [23] they achieved 0.85 accuracy using Random Forest algorithm. Similarly, Support Vector Machines (SVM) have been used to model fake news prediction by Horne and Adali [24] achieving between 0.7 and 0.9 accuracy. SVM has also been used to identify misleading headlines by [25] obtaining 0.75 F1-score. Rubin et al. [26] also applied SVM to satire detection based on translated theories, reaching high accuracy (0.9 precision, 0.8 recall, 0.87 F-score), and Kruger and Stendahl [19] achieved 0.7 F1-score with SVM and sequential minimal optimization. Additionally, machine learning has been applied to assess readability in deceptive news detection, with Santos et al. [27] achieving 0.93 accuracy using Recursive Feature Selection and Random Forest. Furthermore, Vieira et al. [ 17 ] used machine learning with Word Mover’s Distance to measure semantic distance based on subjectivity lexicons. Volkova et al. [28] proposed various linguistic markers, ranging from embeddings to connotations, achieving an F1 score of approximately 0.5 on the detection of deceptive news using a Long Short-Term Memory (LSTM) network, which shows a result no better than chance. Similarly, Govaert and Verstraeten ([29]) conducted a study on articles by a specific journalist (Perdiep Ramesar), focusing on stereotypical linguistic markers and employing the Kruskal-Wallis test to compare their performance on deceptive and truthful news. More recently, Lebernegg et al. ([30]) analyzed a range of features, from lexical diversity to emotional language, obtaining an F1 score of 0.85 using XGBoost algorithm.

While the majority of prior research emphasizes English, several relevant studies address disinformation detection in Spanish. For instance, Masip et al. [31] observed a higher frequency of spatio-temporal information in truthful statements and noted the limited accuracy of text-based deception detection alone, suggesting the importance of non-verbal cues. Similarly, Sola et al. [32] developed a novel Spanish corpus of truthful and deceptive testimonies and proposed an integrative methodology analyzing lexical, morpho-syntactic, content, and speech features. Tretiakov et al. [33] focused on detecting misleading information in Spanish across various sensitive topics like COVID-19, the Catalan crisis, migration, and hate speech, achieving 0.88 accuracy using a fine-tuned BERT model based on stylistic features, context, and complexity.

Nevertheless, prior research has analyzed linguistic markers in various ways. However, subjective markers have been largely overlooked, representing a gap in the current research. It is necessary to bridge disciplinary separations and establish a continuum, exploring how the sender’s imprint is manifested in discourse, examining the application of NLP techniques for extracting pragmatic information, and considering the integration of subjectivity analysis in disinformation detection.

3. Main Hypothesis and Objectives

The main hypothesis of this PhD thesis is based on the viability of extracting linguistic markers indicative of a text’s deceptiveness, focusing attention on the analysis of linguistic elements in the pragmatic dimension, that is to say, the words and structures through which the author’s intentions and point of view are conveyed. Subsequently, the primary objective is to automatically detect deceptiveness in written Spanish news by extracting content-based features that capture the linguistic imprint of the author. The convergence of linguistics and psychology rises from the fact that while linguistic elements are tangible textual components, their conveyed intention and pragmatic force extend beyond only content-based analysis. This integrative approach is necessary because disinformation encompasses both physical content (text, images, video) and nonphysical content (emotions, opinions, feelings), as highlighted by Zhang et al. [34].

On this line, we present specific objectives listed below:

O1. Comprehensive Analysis and Categorization of Linguistic Cues for Disinformation Detection: To systematically compile and analyze linguistic cues present in deceptive texts that allows us to categorize and organize the types of linguistic markers employed for the detection of disinformation, drawing upon the previous studies presented in Section 2.

O2. In-depth Investigation of Objectivity and Subjectivity in Disinformation: To conduct a profound study of the linguistic implications of objectivity and subjectivity principles to develop a methodology to determine the degree of deceptiveness in news articles.

O3. Review of Spanish Datasets For the Development of a Specific Disinformation Dataset: To review datasets in Spanish previously used for the task of automatic disinformation detection. As this work is focused on subjectivity analysis to detect disinformation, it will be necessary to compile and develop a new dataset focused on this task, or to annotate existing datasets to adapt them to the task of automatic disinformation detection.

O4. Development and Evaluation of Explainable NLP Methodologies for Disinformation Detection: To extract and analyze the methodologies employed in prior NLP tools for deception or disinformation detection, starting with ML approaches.

O5. Comparative Evaluation and Iterative Improvement: To obtain empirical results and rigorously compare them with the current state of the art in disinformation detection. This objective also includes the identification of weaknesses in our approach and the implementation of targeted improvements.

O6. Scientific Dissemination of Research Outcomes: To actively disseminate the research processes and findings obtained throughout the development of this PhD thesis to the scientific community.

4. Methodology

Our multidisciplinary methodology integrates analytical and empirical approaches to investigate deceptiveness, drawing from both disinformation and deception detection research. The analytical component involves a comprehensive discourse examination to identify subjectivity markers indicative of deceptiveness, aiming for a deeper understanding of relevant linguistic features for NLP. Complementarily, the empirical component focuses on implementing and evaluating NLP techniques, with particular attention to the explainability of their detection processes for user comprehension. This dual approach seeks to build a robust, empirically validated theoretical foundation for deceptiveness detection in texts.

In alignment with the objectives outlined above, our methodology is structured into distinct stages. As detailed below, each stage will be guided by specific starting points to ensure our research builds upon and contributes meaningfully to prior eforts in this area of investigation:

Stage 1. Classification of Previously Used Linguistic Cues: To develop Objective 1 we compile and analyze previously used linguistic markers we categorize the features employed in the reviewed studies into three primary groups: strictly verbal, subjective, and supplementary features. • Strictly verbal features: These features involve quantifiable text characteristics like word count, sentence length, and readability scores, as stylometric analysis. • Subjectivity features: To identify disinformation, these features analyze how a creator’s subjective opinions are expressed through language, focusing on modalization markers. This includes specific word choices (e.g., adverbs, adjectives, verbs) and syntactic structures that signal the creator’s stance towards the content. These features can indicate engagement, lack of specificity, uncertainty, and other subtle cues potentially revealing a deceptive “imprint”. • Supplementary features: In the context of content-based disinformation detection, this category includes external factors influencing information creation and dissemination, such as the source of the news, the platform it’s shared on, the intended recipients, the subject matter, and the language of the text.

Stage 2. Delimitating the Theoretical Boundaries Between Objectivity and Subjectivity: Focusing on objectivity and subjectivity research, to develop Objective 2 we establish a starting point, the theoretical framework of Tuchman [ 9 ]. In this work the concept of objectivity is presented as part of a "strategic ritual" in news, highlights the inherent aspiration towards objectivity in journalistic practice. This emphasis on objectivity makes the precise identification of subjectivity patterns and forms a crucial step in discerning deceptive news, as deviations from this norm can signal intentional manipulation.

Stage 3: Reviewing, Evaluating and Creating Datasets for Disinformation Detection in Spanish: In order to develop Objective 3 we review previous resources in Spanish for disinformation detection specifically, or false information detection in general, a relevant starting point is the shared tasks often included in conferences to evaluate NLP capabilities.

Stage 4: Familiarization and Evaluation of Machine Learning Techniques for Automatic Disinformation Detection: As our research focuses on linguistic analysis of disinformation, we will be able to analyze whether NLP techniques applied to the detection of false information in general can also be used for disinformation. To develop Objective 4 we will initiate this phase of the research by employing machine learning techniques, as these algorithms ofer a high degree of transparency and ease of manipulation, ensuring a human-in-the-loop approach, and promoting the explainability of these algorithms. It is particularly important to highlight the IberLEF shared tasks on fake news detection in Spanish. Although these tasks focus on general fake news classification rather than the specific goal of disinformation detection, they are vital for advancing Spanish NLP resources and models. During 2020 and 2021, IberLEF proposed shared tasks related to fake news detection. Firstly, MEX-A3T [35] featured tasks on aggressiveness and fake news detection. The top-performing team in fake news detection employed a hybrid method combining a supervised autoencoder with input features derived from BERT encodings and word/character n-grams [36], achieving a macro-F1 score of 0.85. After that, in 2021 it was proposed FakeDeS [37]. This edition focused solely on fake news detection. The leading team also utilized the BERT model [38], attaining a macro-F1 score of 0.76.

Finally, Objectives 5 and 6 will be developed using an appropriate scientific methodology to evaluate the results and disseminate the knowledge generated in this area, contributing to the communication of the research in relevant environments that facilitate knowledge exchange and enrich the investigation.

5. Research issues to discuss

Having established the background, justification, and methodological approach of our PhD thesis, it is now crucial to define a series of research questions that will guide its development and direct our research advancements. The formulation of these questions is a key step for the systematic literature review, as highlighted by Shafril et al. [39], and we therefore propose the following four: • RQ1: Can we develop a methodology for detecting disinformation based on subjectivity markers, considering the traditionally objective nature of the news genre? Numerous studies have demonstrated the applicability of linguistic elements (such as word quantity, verb tense, pronoun usage, and syntactic patterns) on discerning truthfulness or falsehood in written texts, showed by Zhou et al. [ 13 ], DePaulo et al. [ 10 ] or Tretiakov et al. [33], among others. However pragmatic elements such as subjectivity markers have been slightly studied. As we introduced before, some studies are related to subjectivity analysis on news, showed by Shokri et al. [18], Brunato et al. [21] and Ott et al. [20], among others. These studies demonstrate that extracting features related to subjectivity allows for the diferentiation of writing styles in false information detection. Furthermore, this approach has not been applied to disinformation in general, and specifically not to disinformation in Spanish. Moreover, the automation of subjectivity analysis processes requires further attention. • RQ2: Recognizing the inherent limitations of relying solely on linguistic features for identifying disinformation, could incorporating theoretical frameworks from cognitive sciences ofer more efective detection methods? While current NLP techniques provide a foundation for false information detection, and psychological research examines deception through the lens of false testimony, existing approaches often prioritize external fact verification or the analysis of surface-level linguistic structures (morphology and syntax). A promising route of research lies in understanding the underlying cognitive processes involved in the creation of deceptive information before it is encoded into language. By deconstructing these cognitive stages we aim to develop cognitively-oriented methods for the automatic detection of disinformation that go beyond superficial linguistic analysis. • RQ3: Is there a relationship between the level of subjectivity in disinformation and its readability? While prior research has examined readability in the context of fake news [27], the specific readability levels of disinformation remain unexplored. The hypotheses to be addressed in this line of research may be related to: (1) the increased text readability often correlates with simpler structure, a characteristic typically associated with objective writing and, (2) given that subjectivity in disinformation might serve to obscure the act of lying, such texts could potentially exhibit lower readability. • RQ4: Can the detection of disinformation particularly help vulnerable populations, such as those with cognitive disabilities, in avoiding manipulation when they face challenges in independently verifying the truthfulness of information? As highlighted by Moreda et al. [40] in the CLEAR.TEXT project, ensuring accessible written information for everyone is crucial to mitigate the risk of exclusion for individuals with cognitive disabilities.

Acknowledgments

This research is funded by a grant for the recruitment of predoctoral research staf (CIACIF/2023/106) from the Fondo Social Europeo Plus of Generalitat Valenciana - European Social Fund Plus of the Generalitat Valenciana. This research work is part of the project “NL4DISMIS Natural Language Technologies to address disinformation and misinformation” (CIPROM/2021/021) (funded by the Generalitat Valenciana (Conselleria d’Educació, Investigació, Cultura i Esport)), and the I+D project “CORTEX: Conscious Text Generation” (PID2021-123956OB-I00) (funded by MCIN/ AEI/10.13039/501100011033/ and by “FEDER Una manera de hacer Europa” - ERDF A way of making Europe).

Declaration on Generative AI

During the preparation of this work, the author used Gemini in order to grammar and spelling check, Paraphrase, translate and reword. After using this tool, the author reviewed and edited the content as needed and take full responsibility for the publication’s content. [35] M. E. Aragón, H. J. Jarquín-Vásquez, M. Montes-y Gómez, H. J. Escalante, L. V. Pineda, H. GómezAdorno, J. P. Posadas-Durán, G. Bel-Enguix, Overview of mex-a3t at iberlef 2020: Fake news and aggressiveness analysis in mexican spanish., in: IberLEF@ SEPLN, 2020, pp. 222–235. [36] E. Villatoro-Tello, G. Ramírez-de-la Rosa, S. Kumar, S. Parida, P. Motlicek, Idiap and uam participation at mex-a3t evaluation campaign., in: IberLEF@ SEPLN, 2020, pp. 252–257. [37] H. Gómez-Adorno, J. P. Posadas-Durán, G. B. Enguix, C. P. Capetillo, Overview of fakedes at iberlef 2021: Fake news detection in spanish shared task, Procesamiento del lenguaje natural 67 (2021) 223–231. [38] X. Huang, J. Xiong, S. Jiang, Gduf dm at fakedes 2021: Spanish fake news detection with bert and sample memory, Education 6 (2021) 4. [39] H. A. Mohamed Shafril, S. F. Samsuddin, A. Abu Samah, The ABC of systematic literature review: the basic methodological guidance for beginners, Quality & Quantity 55 (2021) 1319–1346. URL: https://link.springer.com/10.1007/s11135-020-01059-6. doi:10.1007/s11135-020-01059-6. [40] P. Moreda, B. Botella, I. Espinosa-Zaragoza, E. Lloret, T. J. Martin, P. Martínez-Barco, A. Suárez Cueto, M. Palomar, et al., Clear. text enhancing the modernization public sector organizations by deploying natural language processing to make their digital content clearer to those with cognitive disabilities (2023).

References

[1] National Statistics Institute of Spain, Science and technology, information society. population using the internet (in the last three months) . type of activities performed online , 2024 . URL: https://www.ine.es/ss/Satellite?L=es_ES&c=INESeccion_C&cid=1259925528782&p= 1254735110672 &pagename=ProductosYServicios%2FPYSLayout, last accessed: March 11 , 2025 .

[2] D. M. J. Lazer , M. A. Baum , Y. Benkler , A. J. Berinsky , K. M. Greenhill , F. Menczer , M. J. Metzger , B. Nyhan , G. Pennycook , D. Rothschild , M. Schudson , S. A. Sloman , C. R. Sunstein , E. A. Thorson , D. J. Watts , J. L. Zittrain , The science of fake news , Science 359 ( 2018 ) 1094 - 1096 . URL: https: //www.science.org/doi/10.1126/science.aao2998. doi: 10 .1126/science.aao2998.

[3] M. Esteban-Bravo , L. D. L. M. Jiménez-Rubido , J. M. Vidal-Sanz , Predicting the virality of fake news at the early stage of dissemination , Expert Systems with Applications 248 ( 2024 ) 123390 . URL: https: //linkinghub.elsevier.com/retrieve/pii/S0957417424002550. doi: 10 .1016/j.eswa. 2024 . 123390 .

[4] E. Saquete , D. Tomás , P. Moreda , P. Martínez-Barco , M. Palomar , Fighting post-truth using natural language processing: A review and open challenges , Expert Systems with Applications 141 ( 2020 ) 112943 . URL: https://linkinghub.elsevier.com/retrieve/pii/S095741741930661X. doi: 10 .1016/j. eswa. 2019 . 112943 .

[5] G. Gravanis , A. Vakali , K. Diamantaras , P. Karadais , Behind the cues: A benchmarking study for fake news detection , Expert Systems with Applications 128 ( 2019 ) 201 - 213 . URL: https: //linkinghub.elsevier.com/retrieve/pii/S0957417419301988. doi: 10 .1016/j.eswa. 2019 . 03 .036.

[6] Directorate-General for Communication Networks, Content and Technology, A multidimensional approach to disinformation. report of the independent high level group on fake news and online disinformation , https://op.europa.eu/en/publication-detail/-/publication/ 6ef4df8b-4cea - 11e8 - be1d-01aa75ed71a1/language-en, 2018 . Accessed on October 24 , 2025 .

[7] L. Yuan , H. Jiang , H. Shen , L. Shi , N. Cheng, Sustainable development of information dissemination: A review of current fake news detection research and practice . systems 2023 , 11 , 458 , 2024 .

[8] X. Zhou , R. Zafarani , A survey of fake news: Fundamental theories, detection methods, and opportunities , ArXiv abs/ 1812 .00315 ( 2020 ). URL: https://api.semanticscholar.org/CorpusID:54437297.

[9] G. Tuchman , La objetividad como ritual estratégico: un análisis de las nociones de objetividad de los periodistas., CIC. Cuadernos de Información y Comunicación ( 1998 ) 199 . URL: https: //revistas.ucm.es/index.php/CIYC/article/view/CIYC9899110199A.

[10] B. M. DePaulo , J. J. Lindsay , B. E. Malone , L. Muhlenbruck , K. Charlton , H. Cooper , Cues to deception., Psychological Bulletin 129 ( 2003 ) 74 - 118 . URL: https://doi.apa.org/doi/10.1037/ 0033 - 2909 . 129 .1.74. doi: 10 .1037/ 0033 - 2909 . 129 .1.74.

[11] V. Hauch , I. Blandón-Gitlin , J. Masip , S. L. Sporer , Are Computers Efective Lie Detectors? A Meta-Analysis of Linguistic Cues to Deception, Personality and Social Psychology Review 19 ( 2015 ) 307 - 342 . URL: http://journals.sagepub.com/doi/10.1177/1088868314556539. doi: 10 .1177/ 1088868314556539.

[12] C. Zhou , K. Li , Y. Lu , Linguistic characteristics and the dissemination of misinformation in social media: The moderating efect of information richness , Inf. Process. Manage . 58 ( 2021 ). URL: https://doi.org/10.1016/j.ipm. 2021 . 102679 . doi: 10 .1016/j.ipm. 2021 . 102679 .

[13] L. Zhou , J. K. Burgoon , J. F. Nunamaker , D. Twitchell , Automating Linguistics-Based Cues for Detecting Deception in Text-Based Asynchronous Computer-Mediated Communications , Group Decision and Negotiation 13 ( 2004 ) 81 - 106 . URL: http://link.springer.com/10.1023/B:GRUP. 0000011944 .62889.6f. doi: 10 .1023/B:GRUP. 0000011944 .62889. 6f .

[14] Á. Almela , A Corpus-Based Study of Linguistic Deception in Spanish, Applied Sciences 11 ( 2021 ) 8817 . URL: https://www.mdpi.com/2076-3417/11/19/8817. doi: 10 .3390/app11198817.

[15] R. Mihalcea , C. Strapparava , Computational laughing: Automatic recognition of humorous oneliners , in: Proceedings of Cognitive Science Conference , Citeseer, 2005 , pp. 1513 - 1518 .

[16] I. Picornell , Cues to deception in a textual narrative context:lying in written witness statements , Ph.D. thesis , 2012 . URL: https://api.semanticscholar.org/CorpusID:141472335.

[17] L. L. Vieira , C. L. M. Jeronimo , C. E. C. Campelo , L. B. Marinho , Analysis of the subjectivity level