The Fourth Edition of the 2024 Workshop on Reducing Online Misinformation through Credible Information Retrieval (ROMCIR 2024) is part of the Satellite Events of the 46th European Conference on Information Retrieval (ECIR 2024). ROMCIR serves as a platform for discussions on accessing accurate information and addressing the issue of information disorder prevalent in today's online landscape. The challenge is multifaceted, encompassing various types of information sources (e.g., websites, social media posts) across diferent platforms and domains (e.g., fake news detection, health-related information retrieval, propaganda reduction). Additionally, there is a critical need to assess the impact of generative models like Large Language Models (LLMs) on inadvertently amplifying misinformation and explore their potential role in supporting Information Retrieval Systems (IRSs). In this context, diverse approaches to the problem of access to truthful information are welcomed. Keynote speech and articles in this year's workshop focus on themes such as health misinformation, multimedia and multimodal fact-checking, and information filtering to combat misinformation.
on the computerization of libraries as follows:
Technology is so much fun but we can drown in our technology. The fog of information can drive out knowledge.
Approximately four decades later, its relevance may arguably surpass its initial significance.
Indeed, the advent of Web 2.0 technologies has ushered in a process of disintermediation in
(M. Viviani)
https://www.iit.cnr.it/en/marinella.petrocchi (M. Petrocchi); http://www.ir.disco.unimib.it/people/marco-viviani/
the creation and dissemination of online content within the Social Web, resulting in
welldocumented challenges such as information overload [
Therefore, the ROMCIR Workshop focuses on studying and developing Information Retrieval
(IR) solutions aimed at providing users with access to relevant and truthful information, while
also addressing the phenomenon of information disorder across various domains. Information
disorder encompasses a spectrum of issues, from unintentional misinformation rooted in
ignorance or bias to deliberate dissemination of false content, both manually and through automated
means [
The resolution of the information disorder issue is inherently complex, involving various
types of content, web platforms, and user objectives. Furthermore, emerging AI-related concerns
such as the explainability of search results [16, 17], assessment of truthfulness in user-generated
content [
The Workshop seeks to explore the challenges surrounding online information disorder within the realm of Information Retrieval, while also delving into associated domains of Artificial Intelligence such as Natural Language Processing (NLP), Natural Language Understanding (NLU), Computer Vision, Machine Learning, and Deep Learning. Therefore, the focal points of interest for ROMCIR 2024 encompass, yet are not confined to: • Artificial Intelligence and information truthfulness assessment • Bot/spam/troll detection • Computational fact-checking/truthfulness assessment • Crowdsourcing for information truthfulness assessment • Disinformation/misinformation and bias detection • Generative models and information truthfulness assessment • Harassment/bullying/hate speech detection • Information polarization in online communities, echo chambers • Propaganda identification/analysis • Retrieval and evaluation of truthful information • Security, privacy, and information truthfulness • Sentiment/emotional analysis and stance detection • Societal reaction to misinformation • Trust, reputation, and misinformation
tificial Intelligence at the University of Santiago de Compostela (Spain).
He received his BS in Computer Science in 1997 and his PhD in Computer Science in 2001, both of them with honors, from the University of A Coruña (Spain). From 2001 to 2002, he was a Lecturer in the San Pablo-CEU University (Spain) and in 2003 joined the University of Santiago de Compostela as a senior research fellow (“Ramón y Cajal” R&D programme). His current research interests include a wide range of Information Retrieval (IR) and related areas such as: IR probabilistic models, summarization, novelty detection, sentence retrieval, patent search and opinion mining. He is an active member of the IR community. He participated in the Programme Committee of prestigious international conferences such as SIGIR or ECIR. He also led several R&D projects and contracts in the area of search technologies. In 2011 he was recognized with an ACM senior member award. Website: https://citius.gal/team/david-enrique-losada-carril/ Health misinformation detection: search challenges, annotation issues and reliability of LLMs. In this presentation, we will share insights from our work at CiTIUS (University of Santiago de Compostela, Spain) on the development of technological and scientific solutions for detecting health misinformation. We will delve into the complexities of developing a multifaceted retrieval system for misinformation detection that integrates multiple content-based features. The challenges of creating robust credibility benchmarks, given the subjective nature of credibility, will also be discussed. Lastly, we will share our recent eforts to evaluate the quality of LLMs’ responses to health-related queries.
The ROMCIR 2024 workshop received 14 submissions, of which 6 were accepted, resulting in an acceptance rate of approximately 43%. The submissions came from five diferent countries, including Germany (2), Italy (1), The Netherlands (1), Poland (1), and Spain (1). This year, submissions have particularly focused on the issues of health misinformation, fact-checking (including multimedia and multimodal approaches), and information filtering and misinformation.
The work by Fernández Pichel, Bink, Losada, and Elsweiler focuses on assessing the credibility of information in the context of health information seeking. The authors acknowledge the subjectivity and bias susceptibility of this process and emphasize the importance of defining robust guidelines for credibility assessment. Through a study involving 1,000 participants, they demonstrate a correlation between participants’ judgments and the reference values established following such guidelines. Further data analyses reveal concerning insights into people’s ability to evaluate the credibility of online medical content, posing the risk of personal harm.
Mongelli, Maiano, and Amerini’s work focuses on enhancing deepfake detection by simultaneously analyzing audio and visual cues, proposing the Convolutional Multimodal deepfake detection model (CMDD). This novel approach improves detection accuracy by leveraging the power of Convolutional Neural Networks (CNNs) to extract spatial and temporal features concurrently. Frick and Steinebach address the challenge of combating false information on social media by proposing a method to assess the check-worthiness of tweets. Their approach incorporates analysis of image content, captions, and text obtained from optical character recognition to outperform existing recognition techniques. Vogel, Möhle, Meghana, and Steinebach’s work focuses on detecting check-worthy statements to prioritize claims for fact-checking. They propose an adapter fusion model combining task and Named Entity Recognition (NER) adapters, achieving state-of-the-art results in check-worthiness benchmarks.
In relation to the last research topic, Hornig, Pera, and Scholtes delve into the issue of propagation of misinformation in the domain of video recommendation. They evaluate a range of top-N recommendation algorithms to assess their efectiveness in minimizing misinformation while optimizing overall performance. Their empirical exploration highlights the potential of certain algorithms, including neighborhood-based, neural, and advanced collaborative filtering approaches, in combating misinformation and promoting responsible recommender systems. Hasimi and Poniszewska-Maranda focus on the broader implications of fake news and disinformation on human rights, particularly freedom of speech. They explore the use of Artificial Intelligence (AI) in detecting and filtering disinformation, highlighting the risks to freedom of expression posed by censorship and the suppression of critical thinking.
The first three editions of the ROMCIR Workshop, all co-located with the ECIR conference, led to fervent discussion and presentation of innovative work concerning a variety of open issues related to information disorder and IR. The first edition took place in online mode on April 1, 2021. The second edition took place both in presence in Stavanger, Norway, and online, on April 10, 2022. The third edition took place in presence in Dublin, Ireland, on April 2, 2023. The papers accepted for the first three editions of ROMCIR are collected in CEUR Proceedings [25, 26, 27], which are freely accessible. Updated information on past and current ROMCIR editions can be found on the oficial website: https://romcir.disco.unimib.it/
Informatics and Telematics of the National Research Council (IIT-CNR) in Pisa, Italy, under the Trust, Security, and Privacy research unit. She also collaborates with the Sysma unit at IMT School for Advanced Studies, in Lucca, Italy. Her field of research lies between Cybersecurity, Artificial Intelligence, and Data Science. Specifically, she studies novel techniques for online fake news/fake accounts detection and automated methods to rank the reputability of online news media. She is the author of several international publications on these themes and she usually gives talks and lectures on the topic. She is CNR lead and WP leader of Humane: Holistic sUpports to inforMAtioN disordErs, sub-project of SERICS (PE00000014), https://serics.eu/en/, NRRP MUR program funded by the EU - NGEU. Website: https://www.iit.cnr.it/en/marinella.petrocchi/
Informatics Systems, and Communication of the University of MilanoBicocca, Italy. He received his M.Sc. and Ph.D. in Computer Science from the University of Milan (La Statale), Italy. He was later a postdoctoral fellow at both Italian (University of Insubria) and foreign institutions (University of Burgundy and INSA Lyon, France). He is involved in organizing several research initiatives at the international level. He was the General Co-chair of MDAI 2019 and organized several Workshops and Special Tracks at International Conferences. He is an Associate Editor of “Social Network Analysis and Mining” (SNAM) and “Frontiers in Artificial Intelligence - Natural Language Processing”, an Area Editor (Web Intelligence and E-Services) of the “International Journal of Computational Intelligence Systems” (IJCIS), and an Editorial Board Member of “Online Social Networks and Media”. His main research interests include Social Computing, Information Retrieval, Natural Language Processing, Privacy, and Trust. On these topics, he has published more than 90 research works in International Journals, at International Conferences, as Monographs and Book Chapters. Website: https://ikr3.disco.unimib.it/people/marco-viviani/ 6.1. Program Committee Members • John Bianchi, IMT Scuola Alti Studi Lucca, Italy • Edoardo Di Paolo, Università degli Studi di Roma “La Sapienza”, Italy • Tiziano Fagni, Istituto di Informatica e Telematica – CNR, Italy • Carlos A. Iglesias, Universidad Politécnica de Madrid, Spain • Udo Kruschwitz, Universität Regensburg, Germany • David Losada, Universidad de Santiago de Compostela, Spain • Lorenzo Mannocci, Università degli Studi di Pisa, Italy • Gabriella Pasi, Università degli Studi di Milano-Bicocca, Italy • Marinella Petrocchi, Istituto di Informatica e Telematica – CNR, Italy • Manuel Pratelli, IMT Scuola Alti Studi Lucca, Italy • Daisy Romanini, Istituto di Informatica e Telematica – CNR, Italy • Paolo Rosso, Universitat Politècnica de València, Spain • Irene Sánchez Rodríguez, IMT Scuola Alti Studi Lucca, Italy • Fabio Saracco, Centro Ricerche Enrico Fermi, Italy • Serena Tardelli, Istituto di Informatica e Telematica – CNR, Italy • Marco Viviani, Università degli Studi di Milano-Bicocca, Italy
Partially supported by re-DESIRE: DissEmination of ScIentific REsults 2.0, funded by IIT–CNR; by SERICS (PE00000014) under the NRRP MUR program funded by the EU - NGEU; by the PNRR ICSC National Research Centre for High Performance Computing, Big Data and Quantum Computing (CN00000013), under the NRRP MUR program funded by the NextGenerationEU; by KURAMi: Knowledge-based, explainable User empowerment in Releasing private data and Assessing Misinformation in online environments, under the PRIN MUR 2022 program (20225WTRFN), https://kurami.disco.unimib.it/. based approach in the covid-19 scenario, Social Network Analysis and Mining 11 (2021) 78. [14] M. Pratelli, F. Saracco, M. Petrocchi, Entropy-based detection of twitter echo chambers,
CoRR abs/2308.01750 (2023). URL: https://doi.org/10.48550/arXiv.2308.01750. [15] M. Mattei, et al., Bow-tie structures of twitter discursive communities, Scientific Reports 12 (2022). URL: https://doi.org/10.1038/s41598-022-16603-7. [16] A. Anand, P. Sen, S. Saha, M. Verma, M. Mitra, Explainable information retrieval, in: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2023, pp. 3448–3451. [17] R. Upadhyay, P. Knoth, G. Pasi, M. Viviani, Explainable online health information truthfulness in consumer health search, Frontiers in Artificial Intelligence 6 (2023) 1184851. [18] M. Soprano, K. Roitero, D. La Barbera, D. Ceolin, D. Spina, S. Mizzaro, G. Demartini, The many dimensions of truthfulness: Crowdsourcing misinformation assessments on a multidimensional scale, Information Processing & Management 58 (2021) 102710. [19] F. Cabitza, D. Ciucci, G. Pasi, M. Viviani, Responsible ai in healthcare, arXiv preprint arXiv:2203.03616 (2022). [20] M. Najork, Generative information retrieval, in: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2023, pp. 1–1. [21] G. Livraga, M. Viviani, Data confidentiality and information credibility in on-line ecosystems, in: Proceedings of the 11th international conference on management of digital ecosystems, 2019, pp. 191–198. [22] G. Livraga, A. Olzojevs, M. Viviani, Unveiling the privacy risk: A trade-of between user behavior and information propagation in social media, in: International Conference on Complex Networks and Their Applications, Springer, 2023, pp. 277–290. [23] C. Lioma, J. G. Simonsen, B. Larsen, Evaluation measures for relevance and credibility in ranked lists, in: Proceedings of the ACM SIGIR International Conference on Theory of Information Retrieval, 2017, pp. 91–98. [24] H. Suominen, L. Goeuriot, L. Kelly, L. A. Alemany, E. Bassani, N. Brew-Sam, V. Cotik, D. Filippo, G. González-Sáez, F. Luque, P. Mulhem, G. Pasi, R. Roller, S. Seneviratne, R. Upadhyay, J. Vivaldi, M. Viviani, C. Xu, Overview of the CLEF eHealth Evaluation Lab 2021, in: K. S. Candan, B. Ionescu, L. Goeuriot, B. Larsen, H. Müller, A. Joly, M. Maistro, F. Piroi, G. Faggioli, N. Ferro (Eds.), Experimental IR Meets Multilinguality, Multimodality, and Interaction, Springer International Publishing, Cham, 2021, pp. 308–323. [25] M. Petrocchi, M. Viviani, Overview of ROMCIR 2022: The 2nd Workshop on Reducing Online Misinformation through Credible Information Retrieval, in: ROMCIR 2022 CEUR Workshop Proc, volume 3138, 2022, pp. i–vii. [26] M. Petrocchi, M. Viviani, Overview of ROMCIR 2023: The 3rd Workshop on Reducing Online Misinformation through Credible Information Retrieval, in: ROMCIR 2023 CEUR Workshop Proc, volume 3406, 2023, pp. i–ix. [27] F. Saracco, M. Viviani, Overview of ROMCIR 2021: Workshop on Reducing Online Misinformation through Credible Information Retrieval, in: ROMCIR 2021 CEUR Workshop Proc, volume 2838, 2021, pp. i–vii.