Research in news recommendation, like other areas of recommender systems, focuses heavily on the utility of recommendations for the consumers of news, that is, the users of apps, sites, or feeds in which personalized news is delivered. This emphasis leaves out an important class of stakeholders, namely professional journalists themselves. We argue that these omissions diminish the value of journalists' work and perspectives and leads to lower quality news delivery. As a possible solution, we show that a multistakeholder approach to news recommendation ofers an avenue for restoring the journalistic voice to news recommendation. In this paper, we argue that news recommender systems should be designed with journalists as first-class users and explore the research implications of such a commitment. We include preliminary results from an interview study exploring the deficiencies journalists find in current news recommendation platforms and what might be the benefits of greater governance over them.
James Carey defines journalism as “carrying on and amplifying the conversation of people
themselves,” emphasizing that journalism is not just about delivering news and information
but also about facilitating public discourse and promoting a sense of connectedness among
people [
For digital journalists, the gatekeeping metaphor obscures five distinct functions, including
intelligent aggregator, community builder, role model, empowerer, and forum leader, in addition
to fulfilling journalism’s three roles of the mirror, the watchdog, and the marketplace of ideas
[
Journalists often blame news aggregator platforms for current economic injustices in the
news media landscape [
Along the way, online news distribution has disrupted traditional news business models.
News organizations are struggling to remain profitable in the face of free, user-generated
content facilitated and recommended on technology platforms. With the profit motive under
threat, there is an increasing emphasis on journalism as a public service that provides social
connection and knowledge to the public [
With some exceptions such as described in [
Baumer and Brubaker [
Provider-side issues in recommendation have been studied from a number of perspectives,
especially in work that examines questions of fairness to providers: see Chapter 6 in [
To the extent that provider-side interaction with recommender systems has been studied, it is often from a critical HCI perspective. A number of researchers have examined how creators and communities develop a variety of reactive practices to cope with the opacity of recommendation algorithms when attempting to understand how their content is distributed [16, 17, 18]. These critical studies ofer a range of perspectives from outsiders attempting to make use of recommendation (especially in social media) to achieve a variety of aims. These perspectives are essential but they do not ofer proposals for the design of systems that incorporate provider objectives.
In this work, we echo the concerns of Stray [19] in noting that collaborations between journalists and technologists are required to bridge the gap between general guidelines about how news recommenders should behave and the technical artifacts needed to achieve the desired goals. The proposal in [19] suggests collaborative creation of artifacts and tools that can become input to technical systems. Our approach is more along the lines of participatory algorithmic governance [20], giving journalists power to directly influence recommendation production.
We propose that news recommendation be reconceptualized as a multistakeholder
recommendation problem [
Multistakeholder perspectives are not new to research in news recommendation. For example, Vrijenhoek et al. [21] examine diferent concepts of the role of media in democratic societies and considers what approaches to diversity are most aligned with each perspective. Given this linkage, the choice of a diversity objective in a news recommender is (implicitly or explicitly) an injection of a journalistic objective in alignment with a particular organizational mission.
Of course, the notion that a recommendation approach focused only on the end user is somehow free of other types of objectives is itself unsustainable when considering fielded commercial applications. A host of decisions about signals of user interest, content quality and freshness, and the weighting of these factors all are inevitably tied to the perspective of the individuals and organizations creating and operating a given recommender system. These are not necessarily journalistic perspectives. Key performance indicators like daily active users (DAU) prominent in industry settings are not necessarily good measures of the usefulness or value of recommendations [22]. So, existing news recommender systems cannot be said to be free of influences from non-user stakeholders. It is just that these influences are rarely acknowledged and not journalistic in orientation.
If we accept news recommendations as potentially a multistakeholder environment and we seek to formalize journalists as first-class users, the following research questions naturally arise: • Input: What type of control do journalists want to exercise over the recommendation of their content? • Output: What type of data do journalists need to receive to understand how the recommender system is handling their content? • Incentive compatibility: In social choice terms, a mechanism is incentive compatible if users do not have an incentive to misrepresent their preferences. Since journalists are efectively competing with each other for the audience, ensuring that the recommendation mechanism is incentive-compatible relative to the journalists’ role is crucial to prevent strategic “gaming”. • Balance: How to balance journalistic objectives against other system goals including efectiveness, personalization, diversity, etc. • Transparency: How to provide insight to readers about how recommendations have been generated in a way that exposes the multiple objectives involved?
Exploring these questions requires capabilities that are just now becoming available in the research community. The NSF-funded project Platform for OPen Recommendation and Online eXperimentation (POPROX), launched in 20231, envisions the creation of research infrastructure for long-term experimentation with multiple aspects of recommendation delivery, set within the news recommendation domain. Once in operation, the platform will enable experimenters to deliver news using custom algorithms and custom presentations to an existing base of users with experimental controls. POPROX will provide the environment in which it is possible to explore a variety of research questions in news recommendations including some of those discussed here.
For the questions above, it is essential to get the perspective of the journalists themselves. Existing news recommendation platforms are not amenable to external control or scrutiny and journalists have not been able to express their voice as to how such platforms should operate. We are in the beginning phases of an interview study to learn about journalists’ perspectives on news recommendations to better characterize this stakeholder perspective and include some initial results in Section 5.
Finally, multistakeholder recommendation presents technical challenges, especially when the number of potential objectives multiplies. Multi-objective optimization methods typically work poorly as the dimensionality of the objectives increases [23]. In addition, optimization assumes the weights or preferences over the tradeofs between diferent objectives are fixed and known in advance, which is not a reasonable assumption in a dynamic setting such as news. We intend to adopt a social choice perspective on a multistakeholder recommendation as implemented in the SCRUF-D architecture [24].
SCRUF-D [24] (and its predecessor SCRUF [25]) are recommendation architectures for
integrating fairness into recommendation generation. SCRUF can be understood as a form of
recommendation re-ranking, one of the most common approaches for fairness-aware
recommendation [
Each agent tracks the level of fairness achieved over a historical time window, relative to a self-defined fairness metric. Historical tracking of the state of fairness gives the model its dynamic character, enabling the system to respond to unfairness generated by a particular sequence of user arrivals. In the compatibility function, each agent also measures the expected propensity of the user to respond to recommendations of sensitive items within that agent’s purview. This capability corresponds to the notion of personalized fairness outlined in [26, 27], where the application of a fairness intervention is tailored to each user’s historical profile. In the second phase, the recommender system and the allocated agent(s) cast ballots, i.e., a ranking / scoring of items, and a preference aggregation mechanism combines them to produce the final list.
This architecture can be extended beyond fairness to any other type of multistakeholder concern that a recommender system might need to incorporate. For example, a journalistic objective around topical diversity can be incorporated into an agent that (instead of tracking fairness) tracks the type of diversity in question and casts its ballots in favor of diverse recommendations. The fact that such concerns will be multiple and potentially in tension is not a limitation. Social choice mechanisms are designed in particular to address problems of conflicting priorities among a variety of agents.
We conducted semi-structured interviews with 9 journalists as part of a larger study to learn about their perspectives on news recommendations. Participants included digital journalists from publications in Colorado and Texas with a wide range of experience from new graduates to veterans with decades of reporting including both staf writers and freelancers. The participants had a wide range of areas of focus including national, local, sports, video and photo journalism as well as editorial and digital specialists. The interviews lasted an average of 60 minutes and were conducted via Zoom. The interview protocol covered the following topics: experiences in journalism, journalistic philosophy, experiences with recommender systems interfaces, challenges and frustrations with news recommender systems, and suggestions to better these systems.
Initial inductive analysis of interview transcripts foregrounds a number of common themes. Here we discuss (dis)empowerment, journalistic priorities, and transparency.
We examined the concerns raised by journalists regarding the impact of recommendation algorithms on their work. Our participants believe that despite producing well-researched and time-intensive articles, journalists often face the frustration of low online readership of their work due to the unpredictability of algorithmic favor. A number of participants lamented the algorithms’ preference for sensational content and viral potential, which skews how journalists are perceived, leading to a disconnect between quality reporting eforts and the digital landscape. Participant 8 described this problem as having an impact on their ability to do reporting: P8: It’s also actively impeding my reporting. Because now, while I’m on the ground and I’m trying to talk to people and I’m just trying to get to this essential information that I hope will help change things and make things better, I have an added barrier because the recommendation algorithms are skewing how they see me.
Journalists noted that their inability to interface directly with news recommendation algorithms leads them and their organizations to a variety of tactics to try to influence engagement metrics, generally known as “search engine optimization” (SEO).
P1: On occasion [there would be] a story that was really well reported, or something that took a lot of time and efort not necessarily performing well online, and that’s really when we would kind of start to mess with the SEO and experiment with the headline, or you know, we would try and send out push notifications or emails at diferent times.
The need to experiment with such tactics, without any guarantee of efectiveness, represents a kind of folk theory response to algorithmic opacity, a phenomenon well-studied in social media contexts [28, 16]. As the participant notes, these responses are experimental and it is dificult to know if they will yield the desired results. Participants also noted that applying such tactics (such as rewriting headlines to attract clicks) may compromise the integrity of the work itself. Yet, journalists and their organizations may feel pushed towards the use of these methods because they lack any other means to influence how their news is recommended.
Journalists find themselves grappling with the limitations of recommendation algorithms as they strive to cover and ensure dissemination of essential stories that matter to their communities. Interviewees were well aware of how algorithmic curation afects article visibility and raised concerns about the consequences of this technology for delivering quality reporting to communities.
Participant 1 was concerned that journalistic quality is not considered in the recommendation ranking.
P1: "It’s not like, there’s anyone at Google sitting around going...’Well, this is important because the quality of this reporting is higher’, it’s just a robot that looks at numbers and then put something in somewhere because of numbers. No one’s actually doing ... qualitative ranking of anything. It’s all based on page views." The format of item ranking itself implies a priority to news stories that the underlying logic does not generally support. Several respondents discussed the problem of using popularity to judge news stories and the bias this introduced into readers’ news diets. Participant 1 also addressed this as follows:
P1: "I wish people knew that when you looked at something that says top stories, top does not mean most important."
Another value that emerged across interviews was the importance of original reporting. Journalists emphasized the crucial role that original reporting (defined as in-depth, well-researched, community-centric news) plays in digital journalism, and noted the challenges they face in delivering such stories to their audiences in the face of algorithmic mediation. Original reporting has a unique ability to provide in-depth context, uncover hidden nuances, and ofer diverse perspectives on complex issues. Journalists believed that such rigor distinguishes quality reporting from lower-quality content (sometimes referred to by participants as “clickbait”), as it empowers audiences to make well-informed decisions and fosters a deeper understanding of the world.
Participant 5, in talking about an ideal news recommender system, made this point: P5: "[I]t would promote the kind of community journalism that serious journalists value. It would act more like a paper of record acts and promote the news that that matters to the communities that it tries to serve."
Participants also stressed that prioritizing original reporting in news recommender systems can foster diversity and equality in the media landscape by allowing underrepresented voices and communities to hear their stories, thereby contributing to a more inclusive and informed society.
Journalists were concerned about user engagement as a measure driving news recommendation, fearing that it causes sensational and entertaining news to be favored over informative, community-oriented pieces. Journalists are concerned about algorithms perpetuating negativity and sensationalism, by promoting articles that elicit strong emotional responses.
P8: "The algorithm seems to like, really put on push for negativity and outrage because it keeps you watching right? And it keeps you engaged..."
A number of our subjects expressed concern about the lack of transparency in news recommender systems, raising questions about their impact on users’ perspectives and information consumption. The journalists have expressed apprehension that algorithms are making choices on behalf of users, potentially creating “echo chambers” and limiting diverse perspectives. Users may unknowingly be led down certain paths based on the top recommendations at a specific moment, which can inadvertently shape their worldview – one participant used the term “brainwashing” to describe this efect. Our interviewees stressed the importance of understanding and critically evaluating these systems, which quite dificult in their current incarnations.
P8: ...the black box of algorithms and recommendations. Because it’s not something that, like a lot of people understand, really... You can’t teach people about it, really, because no one’s going to sit down and like learn all of it.
Another transparency issue that participants discussed was that recommender systems fail to provide users with a full picture of the news. Participant 2 recognized that readers often have limited time to consume news and the convenience of the recommender leads to a lack of understanding of the news environment as a whole.
P2:"I guess my concern is it feels like the they’re making choices for you, and and my concern would be: you may not have time to dig deeper to find the news that you should know, because they’ve given you these 3 top stories, and and that’s all you’ve got time to look at."
Algorithm-driven news recommendation systems have significant implications for news content and journalism. While they ofer eficient ways to deliver personalized content to users, concerns remain regarding the prioritization of sensationalism over substantive reporting, and the potential impact on community journalism and civic discourse. We plan to continue our study with additional interviews and analysis. In our small and preliminary study, journalists do express considerable dissatisfaction with the way that recommender systems present their work and they had ideas about how such systems might do better although they recognized that such improvements might not be possible within the current configuration of news recommendation platforms.
The participants highlighted the need for editorial oversight, that is, involving editors and journalists in curating and selecting news stories to maintain the quality and diversity of content presented to users. They were especially interested in diversifying content to better include local news. They had mixed opinions about personalization itself. Some felt that personalized recommendations were to be avoided; others thought it possible to ofer personalization without compromising on journalistic goals. They also sought transparency both for themselves and for readers.
The news recommendation and social media platforms incorporating recommendation have dramatically changed the experience of news readers and working conditions of professional journalists. A key aspect of this transition has been a transfer of agency around news presentation and prioritization from journalists to algorithmic systems. The work in this paper points the way towards how this system might be rebalanced by treating journalists as first-class users of news recommender systems. We identify key questions, both technical and otherwise, that must be answered in order to do so and outline a research program to address those questions.
In our interviews with journalists, we found that they did not feel that their perspectives were being given weight in the design and operation of news recommender systems. They identified a number of ways in which algorithmic news rankings ignored what they considered to be key aspects of news value and quality. They noted various tricks and work-arounds required to try to influence the recommendation of news stories – mechanisms only necessary because they have no other ways to influence recommendation processes. They also noted the ways in which the lack of transparency in news recommender systems prevents both readers and journalists from understanding how news stories are ranked and distributed. A multistakeholder approach to news recommendation is promising approach to alleviating these concerns. [11] J. Wang, W. Zhang, S. Yuan, Display advertising with real-time bidding (rtb) and behavioural targeting, 2017. arXiv:1610.03013. [12] S. Yuan, A. Z. Abidin, M. Sloan, J. Wang, Internet advertising: An interplay among advertisers, online publishers, ad exchanges and web users, 2012. arXiv:1206.1754. [13] W. Zhang, S. Yuan, J. Wang, Optimal real-time bidding for display advertising, in: Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, ACM, 2014, pp. 1077–1086. [14] B. Edelman, M. Ostrovsky, M. Schwarz, Internet advertising and the generalized secondprice auction: Selling billions of dollars worth of keywords, The American economic review 97 (2007) 242–259. [15] S. Yuan, J. Wang, X. Zhao, Real-time bidding for online advertising: measurement and analysis, in: Proceedings of the Seventh International Workshop on Data Mining for Online Advertising, ACM, 2013, p. 3. [16] M. A. DeVito, D. Gergle, J. Birnholtz, " algorithms ruin everything" # riptwitter, folk theories, and resistance to algorithmic change in social media, in: Proceedings of the 2017 CHI conference on human factors in computing systems, 2017, pp. 3163–3174. [17] S. Bishop, Managing visibility on youtube through algorithmic gossip, New media & society 21 (2019) 2589–2606. [18] E. Simpson, B. Semaan, For you, or for" you"? everyday lgbtq+ encounters with tiktok,
Proceedings of the ACM on human-computer interaction 4 (2021) 1–34. [19] J. Stray, Editorial values for news recommenders, News Quality in the Digital Age (2023) 151–165. [20] M. K. Lee, D. Kusbit, A. Kahng, J. T. Kim, X. Yuan, A. Chan, D. See, R. Noothigattu, S. Lee, A. Psomas, Webuildai: Participatory framework for algorithmic governance, Proceedings of the ACM on Human-Computer Interaction 3 (2019) 1–35. [21] S. Vrijenhoek, M. Kaya, N. Metoui, J. Möller, D. Odijk, N. Helberger, Recommenders with a mission: assessing diversity in news recommendations, in: Proceedings of the 2021 conference on human information interaction and retrieval, 2021, pp. 173–183. [22] J. Stray, A. Halevy, P. Assar, D. Hadfield-Menell, C. Boutilier, A. Ashar, L. Beattie, M. Ekstrand, C. Leibowicz, C. M. Sehat, et al., Building human values into recommender systems: An interdisciplinary synthesis, arXiv preprint arXiv:2207.10192 (2022). [23] Y. Zheng, D. X. Wang, A survey of recommender systems with multi-objective optimization,
Neurocomputing 474 (2022) 141–153. [24] A. Aird, P. Farastu, J. Sun, A. Voida, N. Mattei, R. Burke, Dynamic fairness-aware recommendation through multi-agent social choice, 2023. arXiv:2303.00968. [25] N. Sonboli, R. Burke, N. Mattei, F. Eskandanian, T. Gao, "and the winner is...": Dynamic lotteries for multi-group fairness-aware recommendation, 2020. arXiv:2009.02590. [26] W. Liu, R. Burke, Personalizing fairness-aware re-ranking, arXiv preprint arXiv:1809.02921 (2018). Presented at the 2nd FATRec Workshop held at RecSys 2018, Vancouver, CA. [27] N. Sonboli, F. Eskandanian, R. Burke, W. Liu, B. Mobasher, Opportunistic multi-aspect fairness through personalized re-ranking, in: Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization, UMAP ’20, Association for Computing Machinery, New York, NY, USA, 2020, p. 239–247. URL: https://doi.org/10.1145/3340631. 3394846. doi:10.1145/3340631.3394846. [28] M. Eslami, K. Karahalios, C. Sandvig, K. Vaccaro, A. Rickman, K. Hamilton, A. Kirlik, First i "like" it, then i hide it: Folk theories of social feeds, in: Proceedings of the 2016 cHI conference on human factors in computing systems, 2016, pp. 2371–2382.