The purpose of the IIR workshop is to provide a meeting forum for stimulating and disseminating research in Information Retrieval, where early-stage researchers can network and discuss their research results in an informal way. Since its rst edition in 2010, IIR grew thanks to the support and the cooperation of the Italian scienti c community. After Padua (2010) the workshop took place in Milan (2011), Bari (2012), Pisa (2013), Rome (2014), Cagliari (2015), Venice (2016), Lugano (2017), Rome (2018), and Padua(2019). This volume contains the papers presented at IIR 2021, the 11th edition of the Italian Information Retrieval Workshop held on September 13-15, 2021 at the Department of Electrical and Information Engineering of Politecnico di Bari, Italy. The contributions to IIR 2021 mainly address seven relevant topics: { Search and Ranking: Research on core IR algorithmic topics, including IR at scale. { Domain-Speci c Applications: Research focusing on domain-speci c IR challenges. { Content Analysis, Recommendation, and Classi cation: Research focusing on recommender systems, rich content representations, and content analysis. { Arti cial Intelligence, Semantics, and Dialog: Research bridging AI and IR, especially toward deep semantics and dialog with intelligent agents. { Human Factors and Interfaces: Research into user-centric aspects of IR, including user interfaces, behavior modeling, privacy, and interactive systems. { Evaluation: Research that focuses on the measurement and evaluation of IR systems. { Future Directions: Research with theoretical or empirical contributions on new technical or social aspects of IR, especially in more speculative directions or with emerging technologies.
We received 7 long papers, 5 short papers, and 27 extended abstracts. Both full and short original papers present new research results, whereas extended abstracts contain descriptions of ongoing projects or present already published results. Each submission was reviewed by at least two program committee members. The program also included three invited talks and two tutorials by researchers who have contributed at the national and international level.
The rst keynote titled "Bad Practices" in the Evaluation of Recommender Systems by Paolo Cremonesi (Politecnico di Milano) focused on several \bad practices" in the evaluation procedures of a large set of papers Paolo's research group has analyzed over three years. Some of these issues are already known in the IR community (lack of reproducibility), others are unexpected (errors and questionable choices in the evaluation procedure) and worryingly common in their study. The main goal of that research was to assess whether the baselines chosen for comparisons in the original papers were strong enough to con rm the stated progress. In addition to the main research ndings (most of the baselines are weak and the reported progress is phantom), their work had highlighted some (partially) unexpected side outcomes. The focus of that talk was not on the progress of deep learning recommender algorithms and was not on reproducibility issues (both topics have been widely discussed in other venues) although it marginally touched both points. Rather, the focus of the presentation was on the description of the bad practices detected during these years of experiments, along with an analysis of possible causes and possible remedies.
The second talk is titled Domain-speci c knowledge graphs construction: challenges and opportunities by Omar Alonso (Instacart) described some of the challenges and opportunities when designing and implementing a domain-speci c knowledge graph from the ground up. Although there is a lot of interest in knowledge graphs as a rich structure that can be used in many IR applications like search, recommendations, and question-answering, there is little information on the practical aspects of design and construction. This keynote tried to ll this gap.
The third talk is titled Taming the untameable: How Information Retrieval can be integrated into neuroscience to foster a naturalistic paradigm shift by Elvira Brattico (Aarhus University, Denmark and University of Bari Aldo Moro) de ned a bridge between IR and neurosciences. In particular, the talk showed the use of methodologies for moving from arti cial stimulation paradigms, typically used in neuroscience for maintaining control over manipulated variables, towards a naturalistic paradigm where variables are both well-controlled and closely matched to real-life conditions. The potential of this naturalistic paradigm is becoming evident to the cognitive neuroscience community, also in relation to clinical applications, although the use of IR to inform brain signals remains still mainly con ned to music and sounds. Further avenues of applications can be pursued by fostering new interdisciplinary contaminations.
The two tutorials were held by Nicola Tonellotto and Elisabeth Lex and Markus Schedl.
The former titled IR from Bag-of-words to BERT and Beyond with PyTerrier focused on advances from the natural language processing community that have recently sparked a renaissance in the task of adhoc search. Particularly, large contextualized language modeling techniques, such as BERT, have equipped ranking models with a far deeper understanding of language than the capabilities of previous bag-of-words (BoW) models. Applying these techniques to a new task is tricky, requiring knowledge of deep learning frameworks, and signi cant scripting and data munging. In this full-day tutorial, background on classical (e.g., BoW), modern (e.g., Learning to Rank), and contemporary (e.g., BERT, doc2query) search ranking and re-ranking techniques was provided. Going further, Nicola detailed and demonstrated how these can be easily experimentally applied to new search tasks in a new declarative style of conducting experiments exempli ed by the PyTerrier and OpenNIR search toolkits.
The second tutorial named Psychology-informed Recommender Systems by Elisabeth Lex and Markus Schedl focused on the connection between recommender systems and psychological aspects. In particular, personalized recommender systems have become critical means to support human decision making in today's online world. Most of today's recommender systems are data-driven and exploit behavioral data to learn user models and predict user preferences. While such systems can produce useful recommendations, they do not incorporate the underlying psychological reasons for user behavior in the algorithms' design. The aim of this tutorial was to present recent work in psychology-informed recommender systems that leverages psychological constructs and theories to model and predict user behavior and improve the recommendation process. In the tutorial, Elisabeth and Markus discussed three categories of psychology-informed recommender systems: cognition-inspired, personality-aware, and a ect-aware recommender systems. The tutorial was concluded with grand challenges and potential research tasks for future work.