The 2023 Workshop on Reducing Online Misinformation through Credible Information Retrieval (ROMCIR 2023) is at its Third Edition, as part of the Satellite Events of the 45th European Conference on Information Retrieval (ECIR 2023). ROMCIR aims at ofering a discussion forum about access to truthful information and mitigation to the information disorder phenomenon, which characterizes our current online environment. This problem is very broad, as it concerns diferent information objects (e.g., Web pages, online accounts, social media posts, etc.) on diferent platforms, and diferent domains and purposes (e.g., detecting fake news, retrieving truthful health-related information, reducing propaganda and hate-speech, etc.). In this context, all those approaches that can serve, from diferent perspectives, to tackle the truthful information access problem, find their place. In particular, this year keynote speeches and articles have been presented discussing the problem of providing skills for people to identify the truthfulness of information, preventing access to health misinformation, and improving the evaluation processes of truthful information retrieval solutions.
Technology is so much fun but we can drown in our technology. The fog of information
can drive out knowledge.
ifnd information that is truly useful for their purposes [
This topic is extensive, as it concerns diferent contents (e.g., Web pages, news, reviews, medical information, online accounts, etc.), diferent Web and social media platforms (e.g., microblogging platforms, social networking services, social question-answering systems, etc.), and diferent purposes (e.g., identifying false information, accessing information based on its truthfulness, retrieving truthful information, etc.).
Within the ECIR conference, the key goal of the Workshop is to encourage a discussion between
researchers, also belonging to diferent disciplines, and propose innovative approaches, to
the problem of guaranteeing users access to truthful information that does not distort their
perception of reality, through Information Retrieval solutions [
The problem is still of great interest with respect to many research directions, such as the
access to and retrieval of truthful information, the early detection of dis-/mis-/mal-information,
the development of solutions that can be understood by final users (explainable AI) [
All those approaches that can serve, from diferent perspectives, to tackle the truthful information access problem, find their place in ROMCIR 2023. Specifically, the topics of interest include, but are not limited to: • Access to truthful information • Bias detection • Bot/spam/troll detection • Computational fact-checking • Crowdsourcing for information truthfulness assessment • Disinformation/misinformation detection • Evaluation strategies to assess information truthfulness • Fake news/review detection, deep fakes • Harassment/bullying/hate speech detection • Information polarization in online communities, echo chambers • Propaganda identification/analysis • Retrieval of truthful information • Security, privacy, and information truthfulness • Sentiment/emotional analysis • Societal reaction to misinformation • Stance detection • Trust and reputation Both theoretical studies, model-driven, and data-driven approaches, supported by publicly available datasets are more than welcome.
The ROMCIR 2023 Workshop received 9 submissions, of which 5 were accepted, so with an acceptance rate of around 55%. Articles have been submitted from seven diferent countries, i.e., Germany, Italy, the Netherlands, Spain, Switzerland, Turkey, and the USA. The accepted articles, collected in these Proceedings, have primarily considered three issues from distinct points of view. The first issue concerns the usage of NLP techniques for reducing misinformation, especially toxic content; the second issue concerns children’s right to have access to truthful information; the third issue is focused on distinct aspects of health misinformation detection.
With respect to the first issue, in the article by Tran and Kruschwitz, entitled: “ Towards Reducing Misinformation and Toxic Content Using Cross-Lingual Text Summarization”, the authors address the problem by applying both extractive and abstractive text summarization methods, so they can process documents of any length, and by incorporating machine translation as part of their overall architecture. They consider misinformation to be just one of many types of content that should be automatically identified on the road to a healthier digital ecosystem, and consider toxic content, such as hate speech, as naturally falling within the same scope of their work. On several benchmark collections covering both misinformation and toxic content, they show that their approach is robust and achieves competitive performance on these datasets.
With respect to the second issue, in the article entitled: “How Does Information Pollution Challenge Children’s Right to Information Access?”, by Landoni et al., the authors discuss how information pollution afects a critical but understudied user group: children. They emphasize the importance of taking into account the unique characteristics of children’s search context, which can be defined in terms of various factors, from children’s age, abilities, skills, and cognitive development to the blurred line between learning and enjoyment. In addition, they describe the importance of good design to assist children in their diferent roles as seekers, so that they can recognize and distinguish harmful content from useful content. Therefore, the authors propose expanding the notion of relevance to explicitly consider young searchers’ demand for useful, readable, safe, and reliable online information, which is even stronger than that of adults. The authors then discuss guidelines for efectively involving teachers, parents, and children in the design, introduction, and use of research tools to help young users not only access the information available online, but also benefit from it and learn safely. Focusing on the child not only helps progress in helping a target group but, more importantly, is an excellent starting point for exploring a wide range of issues related to information pollution.
With respect to the third aspect, that of health misinformation, three articles were submitted.
The first, entitled: “ Toxicity and Networks of COVID-19 Discourse Communities: A Tale of two
Media Platforms”, by DiCicco et al., compares and analyzes toxicity between Twitter and Parler
for COVID-19 discourse. In particular, highly toxic individuals and their networks are analyzed
from January 1, 2020, to December 31, 2020. As an outcome of this work, the authors found
evidence that Twitter contained a higher level of toxicity regarding COVID-19 discourse than
Parler. When analyzing COVID-19 vaccine discussions within the Twitter network, prominent
conspiracy theory themes emerged among highly toxic users. Within the Parler COVID-19
vaccine discussion, they identified clusters of highly toxic users and important bridges aiding the
spread of misinformation. These toxic conversations could impact the public health response to
various non-pharmaceutical interventions. The interest in continuing, therefore, research in this
area, is also confirmed by the second paper, which also addresses the problem of misinformation
in COVID-19. It is entitled: “A Fact Extraction and Verification Framework for Social Media Posts
on Covid-19”, by Temiz and Taşkaya Temizel. The authors propose a framework for zero-shot
fact extraction and verification for informal user posts on COVID-19 against medical articles.
The framework includes five main steps: pre-processing of user posts, extraction of claims,
extraction of documents and evidence, and verdict assignment. The framework aims to classify
user posts while presenting the relevant set of evidence extracted from peer-reviewed medical
articles on each assertion in user posts, making it interpretable for end users. The proposed
framework achieves stable and equal performance compared to state-of-the-art supervised
techniques for classifying user posts (CoAID) and rumors collected from social media
(COVID19 Rumor Dataset). Using the zero-shot capabilities of models in the literature, it achieves
superior performance in detecting newly emerged misinformation posts and topics. Finally, in
the third article, entitled “Improving the Reliability of Health Information Credibility Assessments”,
by Fernández Pichel et al., the authors analyze the process of creating training and testing
collections for retrieval systems and analyze the issues involved. This process in fact often relies
on annotations produced by human assessors following a set of guidelines. Some concepts,
however, are prone to subjectivity, which could limit the usefulness of any algorithm developed
with the resulting data in real-world applications. One of these concepts is credibility, which
is an important factor in users’ judgments of whether the retrieved information helps meet
an information need. Hence, the authors evaluate a number of existing evaluation guidelines
with respect to their ability to generate reliable credibility judgments among multiple assessors.
Reasons for disagreement are identified and guidelines are proposed to create a tractable and
actionable annotation scheme that leads to greater reliability among annotators and can inform
why an evaluator made a particular credibility judgment. Promising evidence illustrates the
robustness of the new guidelines, which could be a valuable resource for building future test
collections for misinformation detection, or complementing existing ones [
As part of the Workshop, two Keynote Speeches were given. The first, entitled “A Multidisciplinary Approach to Tackling Online Misinformation”, was given by Udo Kruschwitz. The second, entitled “Evaluating Misinformation: Accounting for Credible and Correct Information”, was given by Maria Maistro. Further details in the following.
Abstract: Online misinformation has become a serious problem and judging by the rapid progress being made in automatically producing highly fluent and well-contextualized text by the press of a button it is fair to assume that the problem is not just not going away but going to get a lot worse. What’s more, it does not just afect individuals but has already been demonstrated to have wider implications on society, just think about some of the disinformation campaigns that had been conducted ahead of key elections. How should the problem be addressed? There is no silver bullet, and what is needed is perhaps a wide range of approaches. The scope of the workshop is defined as exploring the use of credible information retrieval as one way forward, i.e., making sure that users get access to information that is topically relevant and truthful. I will look at a complementary, educational approach that is aimed at equipping the information consumer with the skills and knowledge to deal with misinformation. I will report on various directions and ideas we are exploring in the multinational and multidisciplinary COURAGE project,1 whose ultimate goal is to develop a social media companion aimed at supporting and educating users in dealing with misinformation and other social media threats, efectively teaching them social media literacy skills.
versity of Regensburg. Prior to that, he had worked in the School of Computer Science and Electronic Engineering at the University of Essex for over 20 years (which is where he completed his Ph.D.).
His main research interest is the interface between information retrieval (IR) and natural language processing (NLP). He is particularly interested in projects that are aimed at transferring knowledge from academia into industrial applications. He has been involved in various forms of industry collaborations and is particularly happy about the collaboration with Signal AI which started as a Knowledge Transfer Partnership (KTP) project. The company has since become a key player in AI with more than 200 employees. He is also actively involved in the British Computer Society’s Information Retrieval Specialist Group currently serving as its chair and co-organizes various events such as the Data Science @ Regensburg Meetup. Web site: https://www.uni-regensburg.de/language-literature-culture/ information-science/team/udo-kruschwitz/index.html
Padua, 2011; MSc, University of Padua, 2014) and then Computer Science (Ph.D., University of Padua, 2018). She is a tenure track assistant professor at the Department of Computer Science, University of Copenhagen (DIKU). Prior to this, she was a Marie Curie fellow and a postdoctoral researcher at the Department of Computer Science, University of Copenhagen (DIKU), and at the University of Padua in Italy. She conducts research in information retrieval, and particularly on evaluation, reproducibility and replicability, click log analysis, expert search, and applied machine learning. She has already co-organized several international scientific events and she has served as a member of program committees and reviewer for highly ranked conferences and journals in information retrieval. Web site: https://di.ku.dk/english/ staf/?pure=en/persons/641366
The ROMCIR 2023 Organizing Team was composed of the following people with respect to their distinct roles: • Two Co-chair Workshop Organizers; • One Publicity and Proceedings Chair; • Thirteen Members of the Program Committee.
formatics and Telematics, part of the National Research Council (IITCNR), Pisa, Italy, under the Trust, Security and Privacy research unit.
She collaborates with the Sysma unit at School for Advanced Studies IMT Lucca, in Lucca, Italy. Her field of research lies between Cybersecurity and Data Science. She studies novel techniques for online fake news/fake accounts detection. She is in the core team of the TOols for Fighting FakEs (TOFFEe) project, funded by IMT, and Principal Investigator for the CNR unit of HUMANE (Holistic sUpports against inforMAtioN disordEr), Spoke 2 in SERICS (PE00000014), under the MUR National Recovery and Resilience Plan funded by the European Union - NextGenerationEU. She is Work Package leader in H2020 Medina, where she studies how to automatically translate NL cloud security requirements to machine-readable, enforceable policies. Web site: https://www.iit.cnr.it/en/marinella.petrocchi/
Bicocca, Department of Informatics, Systems, and Communication (DISCo), Milan, Italy. He works in the Information and Knowledge Representation, Retrieval and Reasoning (IKR3) Lab. He is involved in numerous research initiatives that involve accessing and retrieving information, especially truthful information. He has been Co-chair of several Special Tracks and Workshops at International Conferences, General Co-chair of MDAI 2019, and Co-organizer of the First Edition of the ROMCIR Workshop. He is Associate Editor of Social Network Analysis and Mining (SNAM), Springer-Verlag, and Editorial Board Member of Online Social Networks and Media (OSNEM), Elsevier. His main research activities include Social Computing, Information Retrieval, Natural Language Processing, Text Mining, and User Modeling. On these topics, he has published more than 80 research works in International Journals, at International Conferences, as Monographs, and Book Chapters. Web site: https://ikr3.disco.unimib.it/people/ marco-viviani/
Rishabh Upadhyay, PhD Student. University of Milano-Bicocca, Department of Informatics, Systems, and Communication (DISCo), Milan, Italy. Web site: https://ikr3.disco.unimib.it/people/ rishabh-upadhyay/ 6.3. Program Committee • Yelena Mejova, ISI Foundation, Turin, Italy • Preslav Nakov, Qatar Computing Research Institute, HBKU, Doha, Qatar • Symeon Papadopoulos, Inf. Tech. Inst. (ITI), Thessaloniki, Greece • Gabriella Pasi, University of Milano-Bicocca, Milan, Italy • Francesco Pierri, Politecnico di Milano, Milan, Italy • Manuel Pratelli, IMT School for Advanced Studies, Lucca, Italy • Fabio Saracco, Centro Ricerche Enrico Fermi (CREF), Rome, Italy • Rishabh Upadhyay, University of Milano-Bicocca, Milan, Italy • Arkaitz Zubiaga, Queen Mary University of London, London, UK
We would like to thank the authors of the submitted articles for their interest in the considered problem, the Keynote Speakers for the interest aroused in new research directions, and the members of the Program Committee for their valuable contribution to the success of the ROMCIR 2023 Workshop.
Marinella Petrocchi would like to acknowledge the project SERICS (PE00000014) under the MUR National Recovery and Resilience Plan funded by the European Union - NextGenerationEU, the Integrated Activity Project TOFFEe (TOols for Fighting FakEs) https://tofee.imtlucca.it/, and the IIT-CNR funded Project re-DESIRE (DissEmination of ScIentific REsults 2.0).
Marco Viviani would like to acknowledge the project DoSSIER: DoSSIER Domain Specific Systems for Information Extraction and Retrieval (https://dossier-project.eu/), EU Horizon 2020 ITN/ETN (H2020-EU.1.3.1., ID: 860721).