The 2023 Workshop on NLP applied to MisInformation (NLP-MisInfo 2023) is at its First Edition, held as part of SEPLN 2023: 39th International Conference of the Spanish Society for Natural Language Processing. NLP-MisInfo aims at fostering research both at the theoretical and at the level of practical real-world applications of NLP technologies applied to misinformation mitigation. The workshop aims at bringing together researchers, developers and industries interested in the problem of mitigating misinformation through NLP technologies. We will discuss recent trends and research projects, as well as developments and advances being made in the area of NLP to address the problem of misinformation from diferent perspectives.
The impact of fake news on the global economy, public health and even the creation of panic
in society has been extensively documented in the past few years with countless examples
[
These approaches handled the phenomenon from a validity aspect, where they labelled a
claim as “False“ or “True“ [
In spite of the current advances in the field, we, as a society, must be aware not only of fake
news but also of the agents that introduce false or misleading information, their supporting
media, the nodes they use in social networks, the propaganda techniques they use, their
narratives and their intentions [
Therefore, we must address these challenges, providing new techniques and methods to really identify and describe the orchestrated disinformation campaigns, such as: detecting misinformation; claim worthiness checking, stance detection and verified claim retrieval; models of disinformation propagation, source detection using social network analysis; Identifying its malicious intent: narratives that want to be spread, benefited and injured agents and final goals; etc.
All those approaches that can serve, from diferent perspectives, to tackle the misinformation problem, in general, and by using NLP tools in particular, find their place in NLP-MisInfo 2023. Specifically, the topics of interest include, but are not limited to: • Dataset submissions. Present and describe a dataset related to the topic of the workshop that has been or is being developed. • Projects submissions. Describe ongoing projects within the workshop’s topic, both academic and industrial.
• Original, unpublished contributions are also welcome.
The NLP-MisInfo 2023 Workshop received 7 submissions, of which 6 were accepted. Articles have been submitted from seven diferent countries, i.e., Spain, Poland, United Kingdom, Saudi Arabia, Switzerland, Estonia, and France. The accepted articles, collected in these Proceedings, have primarily addressed two topics. The first issue concerns the usage of NLP techniques for detecting misinformation; the second issue concerns more general approaches based on research projects for addressing the misinformation problems from a multi-perspective approach.
With respect to the first issue, in the article by Álvaro Huertas-García et al., entitled: “Countering Malicious Content Moderation Evasion in Online Social Networks: Simulation and Detection of Word Camouflage” , the authors present a set of resources for addressing the misinformation problem. In particular, it introduces novel methodologies and tools to combat content evasion in multilingual Natural Language Processing on social networks. A unique Python package, ”pyleetspeak”, is developed, ofering a customizable system for simulating multilingual content evasion through word camouflage techniques. The study also presents a synthetic multilingual dataset of camouflaged words, facilitating the training of models for camouflage detection. They show the utility of the tool in improving content moderation, enhancing online security, and serving as a potential data augmentation tool for AI systems.
In this issue, we can find the article entitled: “ELAINE: rELiAbility and evIdence-aware News vErifier” , by Carlos Badenes-Olmedo et al. It presents ELAINE, a hybrid proposal to detect the veracity of news items that combines content reliability information with external evidence. The external evidence is extracted from a scientific knowledge base that contains medical information associated with coronavirus, organized in a knowledge graph created from a CORD19 corpus. The information is accessed using Natural Language Question Answering and a set of evidences are extracted and their relevance measured. By combining both reliability and evidence information, the veracity of the news items can be predicted, which is very promising for the veracity detection task.
With the same objective of improving misinformation detection through NLP techniques, we ifnd the article entitled: “Where Does It End? Long Named Entity Recognition for Propaganda Detection and Beyond“, by Piotr Przybyła and Konrad Kaczyński. They investigate how the extensive span lengths afect the recognition of propaganda, showing that the task dificulty indeed increases with the span length. They also propose a new solution, including an adaptive convolution layer that facilitates the sharing of information between distant words. This allows for improved length preservation without sacrificing overall performance.
Finally, with regard to the first issue, the article entitled: “Google Snippets and Twitter Posts; Examining Similarities to Identify Misinformation” by Saud Althabiti et al., investigates the applicability of Google search and its results as a practical tool for detecting fake news on platforms like Twitter. The research focuses explicitly on comparing Google search result snippets with tweets to assess their similarity and determine if such similarity can serve as an indicator of misinformation. However, the study reveals that the observed similarity between tweets and snippets does not necessarily correlate with news credibility.
With respect to the second aspect, research projects about misinformation, two articles were submitted. The first, entitled: “ERINIA: Evaluating the Robustness of Non-Credible Text Identification by Anticipating Adversarial Actions” , by Piotr Przybyła and Horacio Saggion, presents the ERINIA project. This project is aimed to address the challenges posed by the increasing importance of automatic assessment of text credibility. Text classifiers are commonly used by platforms hosting user-generated content, including social media, to aid or replace human moderation in filtering out text that is undesirable for some reason – bullying, hate speech, fake news, etc. Unfortunately, deep neural networks are known for their vulnerability to adversarial examples, i.e. data instances with small modifications that preserve the original meaning, yet change the prediction of the target classifier. Here we describe the research actions of the ERINIA project, planned to tackle this challenge by assessing the robustness of currently used classifiers in the misinformation context, creating better methods for discovering adversarial examples and detecting machine-generated content.
Finally, the article entitled “HAMiSoN Project” by Anselmo Peñas et al. presents the HAMiSoN project which aims at treating misinformation from this holistic view. The main challenge is integrating the message and the network level. To tackle this challenge, they propose to reveal misinformation’s hidden intents: which agents introduce disinformation in social media, which narratives they use and which concrete aims (such as polarising, destabilising, generating distrust, destroying reputation, etc.). They propose also to identify malicious and harmed agents and provide this information to the final analysts and users in explainable ways. Identifying misleading messages, knowing their narratives and hidden intentions, modelling the difusion in social networks, and monitoring the sources of disinformation will also give us the chance to react faster to the spreading of disinformation.
As part of the Workshop, two Keynote Speeches were given. The first was centred on the industry point of view, i.e. how the industry works to mitigate the misinformation. It was entitled “Understanding the discourse: NLP in the fight against disinformation” , and was given by Carlos Ponce, IT engineer in the well-known fact-checker Maldita.es1. The second, with an NLP point of view, entitled “Fake news and conspiracy theories: distinguishing conspiracy narrative from critical thinking”, was given by Professor Paolo Rosso. Further details are in the following.
Abstract: Disinformation is a global problem: it wins and loses elections, generates fear and distrust in the population, and afects the security and integrity of people. At Maldita.es they know this very well, they have been fighting against it and its efects for years. In this workshop, we will take a practical tour of the workflow and the tools that the Maldita team relies on to stand up to this battle. We will talk about their use of NLP to engage with their audience and monitor public discourse and the -sometimes unfathomable- use cases of Machine Learning in the fight against misinformation and the creation of evidence-based content.
director and development manager at Maldita.es. The Maldita.es Foundation exists to help citizens make decisions with all the verified information in hand and so that they do not miss it in the battle against misinformation. It does this through journalism, technology, education and new narratives. Misinformation afects all strata of society and is present in our daily lives; The Foundation develops tools to combat it and generates information based on evidence so that the diferent actors involved, from legislators to content distribution platforms, including journalists, citizens and governments, have verified data to rely on.
Abstract: The ease of generating content online has increased the amount of harmful information that is published. Disinformation is published mostly on social media and propagated on a daily basis. In this seminar I will try to stress the importance of going beyond the analysis of (i) words, (ii) textual information, and (iii) fake news. In order to do that we should: (i) integrate in the architecture of AI deep learning models emotional signals and psycholinguistics characteristics; (ii) address disinformation detection from a multilingual perspective; and (iii) consider that often fake news could be part of a conspiracy theory and a disinformation campaign. Related to the latter, it is important to be able to distinguish between conspiracy theories and critical thinking. A shared task on this topic will be organised in 2024 at PAN2, both in Spanish and in English, with data from Telegram.
de Valéncia, where he is also a member of the Pattern Recognition and Human Language Technology (PRHLT) research centre.
His research interests are focused on social media data analysis, mainly on fake news and hate speech detection, author profiling, and sarcasm detection.
He has published 50+ articles in journals (34 Q1) and 400+ articles in conferences and workshops; he has an H-index of 69 (source: Google Scholar) and he is in the ranking of the top H-index scientists in Spain (http://www.guide2research.com/ scientists/ES). He has been PI of several national and international research projects funded by EC, U.S. Army Research
Ofice, Qatar National Research Fund, and Vodafone Spain . 2https://pan.webis.de/shared-tasks.html
Currently, he is the PI of the research project XAI-DisInfodemics on eXplainable AI for disinformation and conspiracy detection during infodemics (Spanish Ministry of Science and Innovation), and of the Public Procurement with OBERAXE, the Spanish Observatory on racism and xenophobia of the Secretary of State for Migration. Moreover, he is a member of the EC IBERIFIER project on Monitoring the threats of disinformation (European Digital Media Observatory), the project on Resources and Applications for Detecting and Classifying Polarized Hate Speech in Arabic Social Media (Qatar National Research Fund).
He has been advisor of 26 PhD theses and actually he is the advisor of 8 PhD students. He gave several keynotes (TSD-2020, CICLing-2019 etc.) and has helped organising 30+ shared tasks at the PAN Lab at CLEF and FIRE evaluation forums, SemEval, IberLEF and Evalita on topics such as author profiling (e.g. profiling bots, haters, and fake news spreaders), hate speech detection, irony detection, misogyny, sexism and toxic language identification, as well as of the MAMI shared task at SemEval 2022 on misogyny identification in memes. He has been the chair of *SEM-2015, and organised conferences in Valencia such as CERI-2012, CLEF2013, EACL-2017, and NLDB-2022. He helped as senior chair or track chair in conferences such as SIGIR, ACL etc.
Since 2014 he is Deputy Steering Committee Chair of the CLEF Association. He is also Associate Editor at the Information Processing & Management journal. He gave several tutorials on plagiarism detection at ICON-2010, author profiling at RuSSIR-2014, RANLP-2015, FIRE-2016 and CLiC-it-2018, and harmful information (fake news and hate speech) at CIKM-2020. During the last 10 years, the obtained results in plagiarism detection, irony detection, author profiling, and credibility detection (fake news) were covered by Spanish (El País, ABC, La Vanguardia, El Mundo, El Levante, El Confidencial, Radio Nacional de España, La Cope) and international media (Reforma, Informador, CNN-Español). In 2022 he received the UPV Research Award in the category of Excellent Publication in Engineering and Technology for his work on the automatic identification and classification of misogynistic language on Twitter.
The NLP-MisInfo 2023 Organizing Team was composed of the following people with respect to their distinct roles: • Two Co-chair Workshop Organizers; • Fourteen Members of the Program Committee.
Nacional de Educación a Distancia (UNED), Department of Languages and Informatics Systems (LSI), Madrid, Spain, where he has developed his teaching and research career since 2010. In 2012 he obtained his PhD in Computer Science from Rey Juan Carlos University, where he developed his doctoral thesis as an
cial Master’s Degree in Information Technology and Computer Systems and since 2006 he is a Computer Engineer, both from Rey Juan Carlos University. He is currently a member of the Language Processing and Information Retrieval Research Group of the UNED, as well as the Center for Intelligent Information Technologies and their Applications (CETINIA) of the URJC.
In recent years, his research lines have focused on the areas of misinformation mitigation, fake news detection and stance on social networks, on reputation and trust mechanisms based on opinion systems. He is the author of around 20 JCR-indexed publications and conferences classified as highly relevant and relevant. According to Google Scholar, he has an h-index of 13 with over 470 citations. He has participated in various international and national research projects collaborating with several diferent institutions, focused on the application of artificial intelligence techniques to solve real-world problems. Web site: http://nlp.uned.es/~rcenteno/index.php
track) at IXA Group, part of the HiTZ Centre of the University of the Basque Country UPV/EHU, where he is head of the Text Analysis unit. He got a PhD in Computer Science at City, University of London (2007), and he has since been working on Natural Language Processing at several British and Spanish institutions, including a two-year stint in the industry as a research project director. He has been involved as PI or collaborator in more than 40 research projects funded by the European Commission, UK research councils, Spanish Ministry of Science and Basque Goverment and published in major journals (Artificial Intelligence, etc.) and conferences (ACL, EMNLP, EACL, IJCAI, etc.) related to Artificial Intelligence and Natural Language Processing.
Currently, his research is focused on Computational Semantics and Information Extraction, with a strong focus on multilingual and cross-lingual approaches. He was the creator and main developer of IXA pipes, a set of ready-to-use multilingual tools for linguistic processing. He is also PMC and committer in the OpenNLP project of the Apache Software Foundation. Web site: https://ragerri.github.io/ 5.2. Program Committee • Óscar Araque, GSI, Universidad Politécnica de Madrid (UPM) • Carlos Badenes-Olmedo, Ontology Engineering Group (OEG), Universidad Politécnica de Madrid (UPM) • David Camacho, Applied Intelligence & Data Analysis group, Universidad Politécnica de Madrid (UPM) • Jorge Carrillo-de-Albornoz, NLP & IR, Universidad Nacional de Educación a Distancia (UNED) • Pablo Hernandez, Maldita.es • Manuel Montes, Laboratory of Language Technologies of the Computational Sciences
Department (INAOE), México • Borja Lozano, Newtral • Laura Plaza, NLP & IR, Universidad Nacional de Educación a Distancia (UNED) • Anselmo Peñas, NLP & IR UNED, Universidad Nacional de Educación a Distancia (UNED) • Álvaro Rodrigo, NLP & IR, Universidad Nacional de Educación a Distancia (UNED) • Paolo Rosso, PRHLT Research Center, Universitat Politècnica de València (UPV). • Fernando Sánchez, GSI, Universidad Politécnica de Madrid (UPM) • Estela Saquete, Natural Language Processing and Information Systems Group, University of Alicante • Mariona Taule Delor, CLiC- The Language and Computation Center-CLiC, University of Barcelona
We would like to thank the authors of the submitted articles for their interest in the considered problem, the Keynote Speakers for the interest aroused in new research directions, and the members of the Program Committee for their valuable contribution to the success of the NLPMisInfo 2023 Workshop.
Roberto Centeno would like to acknowledge DeepInfo project (PID2021-127777OB-C22) (MCIU/AEI/FEDER,UE) and the CHIST-ERA HAMiSoN project grant CHIST-ERA-21-OSNEM002 by AEI PCI2022-135026-2 funded by the Spanish Research Agency (Agencia Estatal de Investigación).
Rodrigo Agerri would like to acknowledge the Basque Government (Research group funding IT-1805-22) and the following MCIN/AEI/10.13039/501100011033 projects: (i) DeepKnowledge (PID2021-127777OB-C21) and ERDF A way of making Europe; (ii) Disargue (TED2021-130810BC21) and European Union NextGenerationEU/PRTR. Furthermore, Rodrigo Agerri is funded by the RYC-2017-23647 fellowship (MCIN/AEI/10.13039/501100011033 and ESF Investing in Your Future).