The 9th International Workshop on News Recommendation and Analytics (INRA 2021) was held in conjunction with the 15th ACM Conference on Recommender Systems (RecSys 2021), 27 September - 1 October 2021 in Amsterdam, The Netherlands. The workshop brings together researchers, media companies, and practitioners to exchange ideas about how to create and maintain a trusted and sustainable environment for digital news production and consumption. The news eco-system has experienced drastic changes over the course of the last decade. News consumption has shifted online and increasingly toward social media. Users read news from web portals, news aggregators, and social media at an increasing rate. Intelligent systems have entered the picture to support users in finding relevant content quickly. News recommender systems alleviate the problems of information overload via personalizing news based on users' explicit and latent interests, while also possibly causing new problems, such as overpersonalization, echo chambers, and disinformation spread on social media. Perspectives from multiple disciplines are required to understand and develop solutions for fair, diverse, and trustworthy recommender systems.
– News recommendation in social media, – User behavior analysis and user interest modeling in the news domain, Netherlands (J. A. Gulla) – User modeling and user profiling, – Applications of data mining for personalized search and navigation, – Personalized news user interface and visualization, – Diversity and multiperspectivity in news personalization and recommendation, • News Analytics – News semantics and ontologies, – Adaptive and personalized news summarization, categorization, and opinion mining, – Social Graph and heterogeneous network analysis , – User segmentation and community discovery, – Big data technologies for news streams, – News framing research, – Automated news generation , – News political leaning and tone, – News trends and evolution, • Psychological, Societal, and Ethical Aspects of News Personalization Systems – Privacy and security issues, – Clickbait, fake news, and misinformation detection, – Diversity and fairness of news search/recommendation, – Bias in online news, – Transparency and explainability, and – Emotion and cognition in news reception.
INRA 2021 has received ten submissions. Each paper has been reviewed by at least two reviewers in a single blind reviewing process. Thereof, six papers have been accepted for presentations. Due to the Covid-19 pandemic, INRA 2021 took place with a hybrid setting distributed over two diferent days, 25 and 28 September. In the virtual session (25 September), the workshop program included a keynote speech by Frank Hopfgartner who is a senior lecturer in Data Science at the Information School of University of Shefield. His research to date can be placed in the intersection of information systems (e.g., information retrieval and recommender systems), content analysis, and data science. He has (co-)authored more than 150 publications in above mentioned research fields, including a book on smart information systems, various book chapters, and papers in peer-reviewed journals, conferences, and workshops. He contributed to INRA 2021 with his keynote speech titled “Do you see what I see? Search Engines as a Lens of Life during the COVID-19 Pandemic.” The physical session (28 September) included another keynote speech titled “Diverse News Recommendation at RTL” by Daan Odijk who is a lead data scientist at RTL. He takes on the challenges of a broadcaster, such as delivering content more personally and has extensive experience in news media industries.
INRA 2021 also included a panel discussion session with the topic of “News Personalization in the Age of Fake News and Polarization”. Frank Hopfgartner was the moderator of the panel and, panelists consist of eminent experts and scholars in news personalization research as follows: Damian Trilling is an Associate Professor for Communication in the Digital Society at the Department of Communication Science, University of Amsterdam). He has a background in Communication Science and researches how people use news in the current media environment. Jonathan Stray is a visiting scholar at the Center for Human Compatible AI at UC Berkeley, where he works on the design of recommender systems for better personalized news and information. He teaches the dual master degree in Computer Science and Journalism at Columbia University.
Judith Möller is an Associate Professor for Political Communication at the Department of Communication Science at the University of Amsterdam and an Adjunct Associate Professor at the Department of Sociology and Political Science at the University of Trondheim. She has co-founded the research group on AI and conversational agents at the UvA. Edward C. Malthouse is the Erastus Otis Haven Professor of Integrated Marketing Communications and Professor of Industrial Engineering and Management Science at Northwestern University. His research centers on customer engagement and experiences.
Konstantin Pogorelov works as a postdoctoral fellow at the Media, Simula Research Laboratory in Norway. His research includes various topics in computing in mathematics, natural science, engineering and medicine, artificial intelligence, and distributed computing.
The contributions to the workshop inspect diferent aspects of news recommendation and analytics. The contributions discuss technical aspects of news recommender systems such as news story chain detection, news similarity representations, and automatic classification of the stance of news websites. In addition to the technical aspects, presented works include media bias, news selection study, and news research infrastructure. We believe that this diversity of presented papers in INRA 2021 aligns well with the interdisciplinary nature of news recommender systems. We hope that the research community we try to establish around INRA workshop series will help researchers to develop better solutions with the help of diferent points of views from diferent disciplines. Information about previous editions of INRA can be found on the web page1.
We thank our program committee members for producing outstanding reviews and all the attendees for the feedback and good discussions.
Accepted Papers Submissions underwent a single-blind peer-review. Due to time constraints, we could accept six papers. Here, we list the papers, authors, and the abstract. Predicting Feature-based Similarity in the News Domain Using Human Judgments Alain Starke, Sebastian Øverhaug, and Christoph Trattner When reading an online news article, users are typically presented ‘more like this’ recommendations by news websites. In this study, we assessed diferent similarity functions for news item retrieval, by com- paring them to human judgments of similarity. We asked 401 participants to assess the overall similarity of ten pairs of political news articles, which were compared to feature-specific similarity functions (e.g., based on body text or images). We found that users indicated to mostly use text-based features (e.g., title) for their similarity judgments, suggesting that body text similarity was the most representative for their judgment. Moreover, we modeled similarity judgments using diferent regression techniques. Using data from another study, we contrasted our results across retrieval domains, revealing that similarity functions in news are less representative of user judgments than those in movies and recipes.
How to Efectively Identify and Communication Person-Targeting Media Bias in Daily News Consumption? Felix Hamborg, Timo Spinde, Kim Heinser, Karsten Donnay, and Bela Gipp Slanted news coverage strongly afects public opinion. This is especially true for coverage on politics and related issues, where studies have shown that bias in the news may influence elections and other collective decisions. Due to its viable importance, news coverage has long been studied in the social sciences, resulting in comprehensive models to describe it and efective yet costly methods to analyze it, such as content analysis. We present an in-progress system for news recommendation that is the first to automate the manual procedure of content analysis to reveal person-targeting biases in news articles reporting on policy issues. In a large-scale user study, we find very promising results regarding this interdisciplinary research direction. Our recommender detects and reveals substantial frames that are actually present in individual news articles. In contrast, prior work rather only facilitates the visibility of biases, e.g., by distinguishing left- and right-wing outlets. Further, our study shows that recommending news articles that diferently frame an event significantly improves respondents’ awareness of bias. Semi-automated Identification of News Story Chains: A New Dataset and Entity-based Labelin Method Fatih Gedikli, Anne Stockem Novo, and Dietmar Jannach Automatically deciding if two or more news articles cover the same event—thereby building a story chain is an important problem in news analytics, and knowledge about such story chains can be used, for example, in recommendation scenarios to suggest follow-up news articles. While content analysis on the level of individual news articles and on general news topics is well-studied, research on news story chains is still limited, partly due to the dificulty of manually labeling or automatically detecting story chains in larger collections of news articles. In this work, we present a novel (semi-)automated method based on clustering and Named Entity Recognition for creating a dataset for news story chains. An experimental analysis of our method shows that it is highly efective in correctly detecting unrelated stories and identifying candidates for related stories. Thus it helps to reduce manual labeling eforts by 80% without afecting the quality of the dataset. It can even improve the quality of the dataset as manual work is put into only the potentially relevant cases. As an additional result of our work, we publish a new dataset of Business Energy News which was created with the help of our method to foster research in this area.
Towards an Experimental News User Community as Infrastructure for Recommendation Research Joseph A. Konstan, Robin Burke, and Edward C. Malthouse While substantial advances have been made in recommender systems—both in general and for news—using datasets, ofline analyses, and one-shot experiments, longitudinal studies of real users remain the gold standard, and the only way to efectively measure the impact of recommender system designs (algorithmic and otherwise) on long-term user experience and behavior. While such infrastructure exists for studies within some individual organizations, the extensive cost and efort to build the systems, content streams, and user base make it prohibitive for most researchers to conduct such studies. We propose to develop shared research infrastructure for the research community, and have received funding to gather community input on requirements, resources, and research goals for such an infrastructure. If the full infrastructure proposal is funded, it would result in recruiting a community of thousands of users who agree to use a news delivery application within which various researchers would be install and conduct experiments. In this short paper we outline what we have heard and learned so far and present a set of questions to be directed to INRA attendees to gather their feedback at the workshop.
Information Overload and User Statisfaction: Balance Between Reliance on Recommendations and Deliberate News Selection Zhixin Pu and Michael A. Beam High content relevance of news recommendations is a key factor for personalized user experiences. Personal factors, such as seeking/scanning and goal commitment, also impact information overload and user satisfaction in news recommender systems. This experimental study using data from 669 Amazon MTurk workers tests a theoretical process of news selection in news recommender systems. We manipulated two key elements of the news recommendation system: relevance of news article recommendations and the presence of a search bar. Results indicated that recommendation systems providing more irrelevant news recommendations results in users selecting more irrelevant articles and reporting higher information overload and lower satisfaction. Though we did not find evidence that seeking news with a search bar would positively influence goal commitment compared with those scanning a news recommendation portal, goal commitment significantly impacts news selection and perceived information overload and user satisfaction.
Interactive Visualization of the Polarity-based Stance of News Websites using News Genres Masaharu Yoshioka, Norihiko Tatsunami, Masahiko Itho, Noriko Kando, and James Allan We have been working on a project that aims to characterize the stance of news websites. We are proposing a framework that can classify websites in terms of the relative proportions of positive and negative articles about particular topics. This system can represent the website’s general stance for easily comparable topics such as presidential election but it is more dificult to analyze the news website from the viewpoint of its users’ underlying interests. In this paper, we propose a system for the interactive visualization of a website’s polarity-based stance by representing the users’ interests in terms of news genres and demonstrate the system using articles refereed to Donald Trump in 2016 US presidential election periods.
Organizing Committee • Özlem Özgöbek, Norwegian University of Science and Technology, Norway • Andreas Lommatzsch, Berlin Institute of Technology, Germany • Benjamin Kille, Norwegian University of Science and Technology, Norway • Peng Liu, Norwegian University of Science and Technology, Norway • Zhixin Pu, Penn State University, USA • Jon Atle Gulla, Norwegian University of Science and Technology, Norway Program Committee