News Articles in a Recom mender System

Abstract

Recent developments in artificial intelligence allow newsrooms to automate journalistic choices and processes. In doing so, news framing can impact people's engagement with news media, as well as their willingness to pay for news articles. Large Language Models (LLMs) can be used as a framing tool, aligning headlines with a news website user's preferences or state. It is, however, unknown how users perceive and experience the use of a platform with such LLM-reframed news headlines. We present the results of a user study ( = 300 ) with a news recommender system (NRS). Users had to read three news articles from The Washington Post from a preferred category (abortion, economics, gun control). Headlines were rewritten by an LLM (ChatGPT-4) and images were replaced in specific afective styles, across 2 (positive or negative headlines) x 3 (positive or negative image, or no image) between-subject framing conditions. We found that negatively framed images and text elicited negative emotions, while positive framing had little efect. Users were also more willing to pay for a news service when facing negatively framed headlines and images. Surprisingly, the congruency between text and image (i.e., both being framed negatively or positively) did not significantly impact engagement. We discuss how this study can shape further research on afective framing in news recommender systems and how such applications could Proceedings of the International Workshop on News Recommendation and Analytics co-located with the 2024 ACM Conference on Proceedings

1. Introduction

Recommender systems are an essential tool to deliver news content to people these days. They are typically an integral part of any news website. These systems personalize the news experience, tailoring content to align with users’ preferences and behaviors [ 1 ]. At the same time, large language models (LLMs) more and more find their way into the news production cycle, from the back end to the front end [ 2, 3, 4, 5 ]. For example, in Norway and the UK, LLMs are used to summarize news articles to make them easier to digest and engaging [ 6, 7 ]. These AI-driven approaches can help newsrooms to become more eficient by providing new ways to present information.

Although LLMs developments demonstrate considerable potential, they have yet to be widely implemented across newsrooms, particularly local newspapers, which have struggled to engage readers as efectively as social media platforms in recent years [ 8 ]. However, the introduction of LLMs in the newsroom has shown the potential to assist journalists in enhancing reader engagement [ 9, 10 ]. In this paper, we specifically examine the potential to emotionally reframe news with LLMs to increase user engagement.

The framing of news articles has been the subject to the expertise of journalists. Framing refers to the representation of reality by a group of individuals or organizations, in line with specific beliefs or

CEUR

ceur-ws.org persuasive intents [ 11 ]. This practice is relatively common in political and economics news, where specific angles are attributed to a story [ 10 ], and play a crucial role in shaping public understanding and perception [ 12, 13, 11, 14 ].

Recent years have seen an increase in research on the practice of afective framing, which refers to the strategic use of emotional tone in communication—particularly in news articles—and its impact on reader engagement [ 15, 16, 17, 18 ]. This link between afective reframing in news and the engagement levels of readers in turn taps into their engagement levels and whether they would like to pay for subscriptions to news platforms [19]. A primary limitation to these findings is that it is unclear whether they also apply to a news recommender scenario. Arguably, if a news article already matches one’s preferences, the specific frame might not be the main predictor of news consumption.

This study examines the efects of AI-driven afective framing through text and images in the context of a news recommender system (NRS). We introduce a novel methodology that diverges from the conventional, hands-on framing techniques. By utilizing OpenAI’s ChatGPT-4, a Large Language Model (LLM), we investigate its efectiveness in altering the construction of news narratives in afective news framing. Existing literature suggests that these models can craft detailed narrative frameworks that closely resemble human-authored content in news articles, to the extent that it becomes dificult for readers to discern the origin of the articles [20, 21].

Our research questions are the following: • RQ1: To what extent do diferent afective news frames in text and images afect emotional states among readers? • RQ2: How does image-based and text-based afective re-framing afect the intention to pay for a news service, and should these frames be emotionally aligned (i.e., congruent)? • RQ3: To what extent are the efects of afective reframing on intention to pay mediated by user engagement?

2. Background

The following sections provide an overview of related and relevant work in the field. First, we review the literature on news recommender systems (NRSs), emphasizing their role in mitigating information overload and aligning users’ preferences by tailoring content. Thereafter, we discuss work on afective framing in text and images; these factors afect news frames in shaping readers’ perceptions and behaviors. Finally, we work on how afective framing influences user engagement in news consumption. These subsections provide an understanding of the intersections between recommender systems, afective framing, and user engagement in digital journalism.

2.1. News Recommenders and User Preferences

There is far more news content available online than one can reasonably consume or browse through. This leads to information overload [22, 23], which hinders users’ capacity to identify content relevant to their interests and benefits [ 1 ]. Given this scenario, NRSs are crucial as they filter content and present personalized recommendations, efectively enabling users to navigate the massive online news and information. By tailoring content that aligns with user profiles and preferences, these systems accelerate the information-retrieving process, raising the eficiency and relevance of online news consumption [24, 25, 26]. Existing studies indicate that personalized information systems enhance users’ perceived relevance, involvement, and engagement with content [27, 28]. In addition, NRSs not only facilitate access to content relevant to individual users, but also yield commercial advantages for platform providers [29].

NRSs are geared towards showing content that align with its users’ preferences. These preferences span a wide array of interconnected factors, including but not limited to specific subjects of interest, readers’ emotional states, the credibility of news, users’ attitude [30, 31, 24]. Recommender systems usually rely on ratings to indicate their preferences for items and also gather clickstream data to infer the interests or preferences of users [32, 33, 34]. These systems improve user engagement and satisfaction by ifnely tuning recommendations to align with individual user preferences [ 35, 36]. However, personalized recommender systems may cause the adverse polarization efect from ofering customized content to users via users’ recorded behaviors, preferences, and tendencies [37, 38]. Considering users’ preferences are crucial for the eficacy of NRSs. It’s essential to balance personalization with diversity, ensuring users access to a wide array of information tailored to their interests and requirements [39, 40].

2.2. Afective Framing in Text and Images

2.2.1. Framing.

News media, central to democratic societies, significantly influences public opinion by framing issues and events to define and highlight particular aspects [ 11 ]. To frame means opting for aspects of a perceived reality and making them more prominent within a communicating text. In this way, it promotes a specific definition of the problem, causal interpretation, moral evaluation, and/or treatment recommendation for the issues being discussed [41]. Frames draw attention to specific information related to the subject being communicated. By making certain details more salient, they enhance the likelihood that the audience will notice and understand this information, subsequently processing and storing it in their memory [ 42, 41 ].

Frames are components of political debates, journalistic norms, and discourse in social movements. This is the alternative approach to interpreting and defining issues in the political and social world [ 11 ]. Journalists use news frames to provide interpretation of events and contentious topics. By reducing complex discussions and policy matters to the elements, these frames identify the responsibility for issues and ofer guidance on possible directions for action [ 43 ]. The application of news framing varies depending on the nature of the media outlet and the topic. This variation is less pronounced between diferent media types, such as television versus print, but more significant between categories of news outlets, namely sensationalist versus serious [ 44 ].

For example, one news study on economic issues indicated that afective attributes of the news articles, particularly positive versus negative frames, have a certain influence on people’s evaluation of economics. Specifically, a negative frame can significantly afect readers’ expectations and performance in relation to the economy [ 45 ].

2.2.2. Afective Framing in text and image.

Numerous studies have demonstrated that afective factors in news production, play a significant role in shaping public perceptions. A key element in how the media influences readers’ interpretations of specific events is the use of afective frames, such as the representation of political candidates. The concept of afective framing relates to the emotional tone conveyed in news articles [ 15 ]. Journalism faces uncertainty amid economic, political, and social crises, creating a volatile environment where employing afective framing becomes a strategic tool to steer through and mitigate these challenges [ 46 ]. Afective framing is a spontaneous, non-inferential, and pre-reflective method of sorting and choosing information. This process simplifies complex information to first-personally manageable, giving it a specific cognitive significance. This form of framing emphasizes specific emotions further in the article content [ 47, 48 ]. Based on these, afective framing also afects the reader’s emotional states or attitudes. For instance, one afective framing study found that viewing the negatively framed tweets amplifies the unfavorable emotional state of readers and leads to an increased willingness for environmental protection [ 49 ]. Moreover, another study also indicates that the participants exposed to positively framed messages revealed a more optimistic attitude toward the water recycling issue than those subjected to negatively framed information [ 50 ]. In addition, regarding preference, one study on news frames suggests that individuals interested in political issues are more likely to select negative news stories. The results indicate a common preference among participants for negative news articles [ 51 ]. Therefore, afective framing shapes public perception and opinion. It influences the reader’s emotional states and thus impacts the decision-making process [ 52 ].

Another crucial element in news framing is imagery; which shapes readers’ interpretation of the text by triggering certain cognitive frameworks through associative reasoning [ 53, 54 ]. Images serve as an influential medium, ofering a less cognitively demanding and intrusive experience compared to text. Their visual impact, which closely mirrors reality, has the capacity to evoke strong emotions. For instance, due to their compelling appeal, images often appear on pages and websites, setting the initial context for a story [ 55 ]. It is essential to combine images and text in news-press. According to [ 56 ], traditionally, images were used to illustrate the text, providing a visual representation that the text elaborated on. This study highlights a historical reversal: the text now relies on the image, adding layers of meaning, culture, and imagination to the primary visual message rather than the image just illustrating the words. This nuanced interaction between text and image underlines the complexity of how media construct and interpret meaning in news media. For example, research indicates that emotional responses to pleasing visual slides displayed high levels of positivity and minimal negativity [ 57 ]. One study on environmental protection suggests that environmental organizations can show visual elements that depict the negative impacts of human activity on the environment. This strategy is recommended to more efectively capture public attention [ 58 ]. Another study highlights that the presence of visual elements can influence readers’ willingness to share news, compared to articles that only contain text [ 59 ].

2.3. Afective Framing and Engagement

Afective framing, via both textual and visual elements, is essential in transforming users’ emotions, perceptions, and decision-making processes [ 60, 61, 62 ]; it highlights the importance of how news framing influences user engagement, potentially altering reader engagement through the manipulation of news articles, such as in political contexts [ 63 ]. For instance, one study investigates how positive and negative news framing afect people’s engagement in mobilizing to vote in referendum campaigns, especially for the group that is against the proposals. The result indicates that positive news mobilize skeptics to participate in a referendum vote [ 64 ]. Another study examining the efect between sharing news based on positive and negative news framing indicated that negative fake news increases readers’ willingness to share, while positive fake news does not reach a statistically significant level. Furthermore, negative emotions mediate in the viralization of news content, whereas positive emotions do not have the same efect [ 65 ]. Another study focusing on social media explores the readers’ engagement with news articles characterized by positive and negative emotions such as anger, fear, hope, and happiness. The finding shows a positive correlation between negative news and readers’ likes, shares, and comments on the articles [ 66 ]. Therefore, afective framing alters readers’ emotional state and impacts the decisionmaking process, potentially leading to varied levels of engagement.

Consequently, afective framing significantly influences news engagement, shaping readers’ perceptions and potentially impacting society. With ChatGPT’s swift rise to global, this technology ofers a nuanced understanding of human language [67]. Understanding the complex relationship between afective framing and this advanced AI technology is crucial.

2.4. Contribution of Current Research

We have discussed two key areas of research: NRSs and the role of afective framing in influencing user engagement. Much of the existing recommender research has focused on mitigating information overload in news consumption by personalizing content based on user preferences [ 1, 24, 25, 26 ]. Additionally, research on afective framing has shown its significant impact on readers’ emotions and decision-making processes [ 49, 15, 52 ].

However, there is limited research exploring the interaction between NRSs and afective framing, particularly in the context of news reframing by LLMs. Notably, previous studies have typically focused on either text or image framing independently [ 49, 50, 51, 53, 59 ]. In contrast, our research considers both elements together, investigating how AI-generated afective framing in both text and images influences user engagement and their willingness to pay for news services.

Our contributions to the field are as follows: 1. Integration of Afective Framing and Recommender Systems: We explore how AI-driven afective reframing, using ChatGPT-4, impacts user emotions and perceptions in news recommendations—an area that has not been extensively explored in existing research. 2. Congruency of Text and Image: We investigate the efects of congruency between afective text and images on user engagement and behavior, addressing a key gap in the research on how diferent media cues interact in news consumption. 3. User Engagement and Intention to Pay: Our findings provide empirical evidence on the influence of afective framing on user engagement and their willingness to subscribe to news services, which contributes not only to academia but also to the journalism industry.

3. Methods

3.1. System & Dataset

We developed a research platform to address our research questions. It utilized news articles from Washingtonpost.com/opinions/, a popular commercial news website which features opinion articles. We chose this dataset primarily because it is a well documented and widely used dataset in news RecSys (see, for example, [68] and [69]). This makes the results more reproducible, as other users can also access the news articles and results.

We selected three diferent news topics and sampled 18 articles from each: (1) Abortion: reproductive health and rights, (2) Economy: politics, and (3) Gun control: firearms. The news articles have been specifically selected to ensure that we would be able to perform a valid experiment, to control for diferent user attitudes regarding diferent controversial topics [ 70].

3.1.1. Model selection

For our study, we chose to summarize each of the 18 articles with OpenAI’s GPT-4 model (version gpt-4-0613) to generate two summaries, one with a positive valence and one with a negative valence. We utilized this model due to it’s advanced natural language processing capabilities, suitability for generating contextually relevant text, and tokens limitation. The temperature parameter was set to 0.8 to ensure that the generated text was engaging, while maintaining a balance between creative and coherent headlines that comply with the input prompt. The Top_P parameter used in this experiment is the default parameter of 1.0. This parameter enhances the quality of the output by focusing on a broad range of probable next words while allowing variability and diversity in the output. All articles were written in English1.

Our study analyzed the shift in users’ emotional states, engagement and intention to pay in the news across six conditions. The afective framing for each condition in the study is illustrated in Figure 5, with each representing a distinct combination of text and image. Condition A, B, and C were positive framing, with no, incongruent, and congruent images. By contrast, conditions D, E, and F were negative news framing, with (i) no images, (ii) incongruent and (iii) congruent images. We used JavaScript, HTML, CSS for the frontend, and Django with Python for the backend, integrating Washington Post content. We also conducted a T-test and Structural Equation Modeling (SEM).

3.2. Procedure

The procedure of the study is shown in Figure 1. Participants were invited to join a survey in which they could test out an online news service. After disclosing demographics, topic preference, and current emotional state, participants were presented three article previews in our news service interface based on their topic preferences. The topics included Abortion, Economics, and Gun Control. Please refer to the footnote to inspect the prompts of our study2. For each preview, users were asked to indicate to which extent they would want to read more of the article preview, and would like to share with others. In addition, users had to assess how much they liked the article, the article’s content, and whether the article aligned with users’ general preferences and mood. Following the three previews, participants were then asked to self-report their emotional state when reading the previews, in addition to evaluating to which extent, based on the recommended previews, they would want to pay for access to a similar online news services (see Figure 1). 1This research adhered to the ethical guidelines of the Research Council of Norway and the guidelines of University of Bergen for scientific research. The study was judged to pass without further extensive review, for it contained no misleading information, stress tasks, nor would it elicit extreme emotions 2The prompts we used in ChatGPT-4 to reframe news articles are here: https://anonymous.4open.science/r/ RecSys2024AffectiveReframing-2505/README.md

3.3. Measures

We inquired on one three main dimension in our experiment: (i) users’ Emotional States, (ii) their experienced Engagement with our news platform, and (iii) their Intention to pay of news articles, all on 5-point Likert scales.

3.3.1. Emotional State Scale

Users’ emotional states were measured both pre-test and post-test using the International Positive and Negative Afect Schedule Short-form (I-PANAS-SF) [ 71]. This included ten items split into two subscales: Positive Afect (‘alert’, ‘inspired’, ‘determined’, ‘attentive’, ‘active’) and Negative Afect (‘upset’, ‘hostile’, ‘ashamed’, ‘nervous’, ‘afraid’).

3.3.2. Engagement and Intention to pay

Table 1 describes the items used for engagement and the results of our confirmatory factor analysis. All items were measured on 5-point Likert scales (1 = Strongly Disagree, 3 = Neutral, Disagree, 5 = Strongly Agree). Engagement included items related to willingness to read, liking, preference and mood alignment and sharing intent (adapted from studies cited in [ 1 ]). While we initially aimed to diferentiate between these items and perceived trust (taken from [ 72]), the collapsed into a single factor.

Additionally, we also inquired on a user’s intention to pay for a news service similar to the one used in the study, based on an item from [73]: “Based on the recommended articles, I would want to pay for access to online news services in the future similar to this one”.

4. Results

A total of 300 participants from the USA ( = 40.31, = 12.04 , 50.6% males) completed our user study. All 300 participants were recruited from the crowdsourcing platform Prolific. Our sample size is suficiently large to be able to detect efects that were theoretically and practically meaningful [ 40].

We analyzed pre-post diferences for positive and negative emotional states, across six afective reframing conditions. We also explored how diferent interaction of text, image and news topics afected readers’ engagement and intention to pay for a news subscription in a Structural Equation Model (SEM).

4.1. RQ1: Changes in Emotional States

Overall, as shown in Figure 2, our results indicate that Negative news framed by GPT-4 triggers stronger emotional responses at significant statistical levels, especially when negative text is coupled with congruent imagery, compared with Positive news. Similarly, negative images alter more users’ emotional states [ 49 ].

Regarding positive conditions, emotional changes and responses were less pronounced. The emotions that reached a significant level of mean diference included active and nervous (condition A; no image), the inspired, upset and hostile (condition B; incongruent), and shame (condition C; congruent). This suggested that positive text was less likely to elicit clear changes in a user’s emotional state, regardless of the accompanying image shown.

In contrast, negative conditions exhibited more pronounced emotional changes and responses. Condition D revealed inspired, upset, hostile, and afraid. Condition E displayed emotions such as inspired, upset, hostile, ashamed, and afraid. Finally, Condition F encompassed a range of emotions, including inspired, determined, upset, hostile, ashamed, and afraid.

4.2. RQ2-3: Engagement and Intention to Pay

We submitted all measures and questionnaire items to a structural equation model (SEM) analysis. We ifrst tested a model a fully saturated model, where all condition efects afected engagement and intention to pay and performed stepwise removal of non-significant relations afterwards. The resulting model is depicted in Figure 3, and had excellent fit statistics: 2(56) = 91.976, < 0.01 , = 0.996 , = 0.995 , = 0.027 , 90% − : [0.016, 0.036]. Our path model met the guidelines for discriminant validity, as well as construct validity (cf. Table 1).

We examined whether diferent afective frames and the alignment of afective frames afected engagement and intention to pay. As depicted in Figure 3, we found two efects of our afective frames. First, we found that users facing headlines that were comprised negative language were more likely to pay for a news service based on our system, as positive text negatively afected intention to pay ( = −.434 ). Although no efects were found on user engagement, it did suggest that users would be more likely to use a news service when observing negatively valenced news. Second, we observed an interaction efect between text and image emotions on intention to pay, which was complementary to the main efect of text.

To better understand this interaction efect, please refer to Figure 4. Depicted is a user’s intention to pay accross the six reframing conditions. The graph clearly shows the main efect of average negative text leading to a higher intention to pay. For the interaction efect, there was a significantly higher intention to pay for negative headlines accompanied by a negative image ( ≈ 2.22 ), compared to positive headlines accompanied by a negative image ( ≈ 1.62 ). This suggested that negative emotions in a news article may have a positive efect on a user’s willingness to use that news service.

The aforementioned interaction efect and Figure 4 also shows the role of alignment between text and image emotions. Whereas negative emotions seemed to support each other in eliciting user responses, the combination of a positive headline with a positive image actually led to a weaker user response, compared to, for example, a negative headline with a positive image. This suggested that alignment or congruency between image and text emotions was mostly beneficial for negative emotions.

Regarding further mediation efects, Figure 3 shows no mediated efects from our reframing manipulations. Although we did observe a positive relation between user engagement and intention to pay ( = .633 ), it only acted as a mediator for a news category. As mentioned, all efects directly afected a user’s intention to pay. We did observe that users found news articles in the abortion topic more engaging, when compared to the economics category ( = .475 ). However, we did not observe any significant interaction efects between the news topic and the emotional manipulation, suggesting users found the news topic more engaging, but not in relation to our afective manipulations. Note that we observed no such efect between gun control and economics.

5. Discussion

We have investigated the impact of afective reframing on news consumption by performing Large Language Models (LLMs). Our primary focus was to provide nuanced insights into news recommender systems (NRSs) regarding the emotional states, engagement, and subscription intentions of readers. This is achieved by exploring the intersectional efects of text framed by GPT-4 and imagery, and news topics aligning readers’ preferences. We have gathered valuable insights relevant to the application of Artificial Intelligence in journalism.

The following insights are derived from our analysis of emotional states (RQ1), image and text-based afective reframing, and engagement and intention to pay for news articles (RQ2 and RQ3).

Regarding [RQ1], we find that negative frames substantially trigger and alter readers’ emotions, compared to positive frames. This observation is consistent with prior studies outside of LLMs, which demonstrate that afective framing influences readers’ emotional states [ 49 ].

Regarding RQ2 and RQ3, we found that congruence between negative news frames and images increased readers’ intentions to subscribe but did not significantly afect engagement. This aligns with research indicating that negative news afects user engagement more than positive news [ 65 ], and visual elements enhance engagement more than text-only content [ 59 ]. News topics matching user preferences also boost engagement, supporting findings that recommender systems can increase engagement by aligning content with individual preferences [35, 36].

Our path analysis shows a significant relationship where engagement leads to a greater willingness to pay. While this confirms that engagement predicts subscriptions [ 19], engagement does not mediate our reframing manipulations.

These results contribute the essential consequences for both Newsroom and the development of NRSs. Our research initially depicts that afective frames by LLM model in news articles can significantly trigger emotional reactions. These changes in emotion may influence the audience’s decision making process [75, 76, 77, 78]. Our findings highlight that negative news frames, especially when accompanied by congruent imagery, enhanced user willingness to pay. These supports existing research suggesting that coordinating text and images in news articles influence users’ behavior and perception [ 79, 80], and readers’ preference, engagement and payment intentions are correlated to each other [ 81, 59, 19 ].

6. Conclusions & Future Work

The efects of LLM-driven afective reframing in news are nuanced. It seems that negative frames significantly amplify emotional reactions and increase the intention to pay for news content, especially when paired with congruent imagery. In contrast, incongruency regarding emotions in a news articles may inhibit changes in emotional states, as well as in perceived and experienced user engagement responses.

Our study is subject to limitations. The lack of an actual news platform would have further increase the validity of our findings, as well as if our NRS would have been more extensive. Additionally, our focus on a knowledge-based personalized recommender system may overlook biases from content diversity that could polarize perceptions. Our findings are also limited by only considering The Washington Post in our dataset, which may not represent broader media biases. Future research will examine the impact of afective framing on news consumption and assess subscription intentions from both personalized and varied-content approaches. We will also study complex multi-level interactions and include a wider range of news sources to expand the applicability of our results.

7. Acknowledgments

This work was supported by the Research Council of Norway with funding to MediaFutures: Research Centre for Responsible Media Technology and Innovation, through the Centre for Research-based Innovation scheme, project number 309339. [18] H. S. Kim, Attracting views and going viral: How message features and news-sharing channels afect health news difusion, Journal of communication 65 (2015) 512–534. [19] H. Davoudi, M. Zihayat, A. An, Time-aware subscription prediction model for user acquisition in digital news media, in: Proceedings of the 2017 SIAM International Conference on Data Mining, SIAM, 2017, pp. 135–143. [20] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al., Language models are few-shot learners, Advances in neural information processing systems 33 (2020) 1877–1901. [21] C. Kolo, J. Mütterlein, S. A. Schmid, Believing journalists, ai, or fake news: The role of trust in media., in: HICSS, 2022, pp. 1–10. [22] B. Scheibehenne, R. Greifeneder, P. M. Todd, Can there ever be too many options? a meta-analytic review of choice overload, Journal of consumer research 37 (2010) 409–425. [23] A. D. Starke, J. Sedkowska, M. Chouhan, B. Ferwerda, Examining choice overload across single-list and multi-list user interfaces, in: 9th joint workshop on interfaces and human decision making for recommender systems (IntRS’22), 2022. [24] S. Raza, C. Ding, News recommender system: a review of recent progress, challenges, and opportunities, Artificial Intelligence Review (2022) 1–52. [25] P. Dandekar, A. Goel, D. T. Lee, Biased assimilation, homophily, and the dynamics of polarization,

Proceedings of the National Academy of Sciences 110 (2013) 5791–5796. [26] D. Jannach, M. Zanker, A. Felfernig, G. Friedrich, Recommender systems: an introduction, Cambridge University Press, 2010. [27] M. A. Beam, Automating the news: How personalized news recommender system design choices impact news reception, Communication Research 41 (2014) 1019–1041. [28] E. Pariser, The filter bubble: What the Internet is hiding from you, penguin UK, 2011. [29] X. Wang, C. Liu, et al., Design of personalized news recommendation system based on an improved user collaborative filtering algorithm, Mobile Information Systems 2023 (2023). [30] P. Pu, L. Chen, R. Hu, A user-centric evaluation framework for recommender systems, in:

Proceedings of the fith ACM conference on Recommender systems, 2011, pp. 157–164. [31] J. Liu, P. Dolan, E. R. Pedersen, Personalized news recommendation based on click behavior, in:

Proceedings of the 15th international conference on Intelligent user interfaces, 2010, pp. 31–40. [32] G. Jawaheer, P. Weller, P. Kostkova, Modeling user preferences in recommender systems: A classification framework for explicit and implicit user feedback, ACM Transactions on Interactive Intelligent Systems (TiiS) 4 (2014) 1–26. [33] C. Trattner, D. Elsweiler, An evaluation of recommendation algorithms for online recipe portals,

CEUR Workshop Proceedings, 2019. [34] M. Elahi, A. Starke, N. El Ioini, A. A. Lambrix, C. Trattner, Developing and evaluating a university recommender system, Frontiers in Artificial Intelligence 4 (2022) 796268. [35] D. Kalimeris, S. Bhagat, S. Kalyanaraman, U. Weinsberg, Preference amplification in recommender systems, in: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, 2021, pp. 805–815. [36] S. M. McNee, S. K. Lam, J. A. Konstan, J. Riedl, Interfaces for eliciting new user preferences in recommender systems, in: User Modeling 2003: 9th International Conference, UM 2003 Johnstown, PA, USA, June 22–26, 2003 Proceedings 9, Springer, 2003, pp. 178–187. [37] C. C. Aggarwal, et al., Recommender systems, volume 1, Springer, 2016. [38] J. Stray, Designing recommender systems to depolarize, arXiv preprint arXiv:2107.04953 (2021). [39] M. Elahi, D. Jannach, L. Skjaerven, E. Knudsen, H. Sjøvaag, K. Tolonen, Ø. Holmstad, I. Pipkin, E. Throndsen, A. Stenbom, et al., Towards responsible media recommendation, AI and Ethics (2022) 1–12. [40] A. Starke, E. Asotic, C. Trattner, “serving each user”: Supporting diferent eating goals through a multi-list recommender interface, in: Proceedings of the 15th ACM Conference on Recommender Systems, 2021, pp. 124–132. [41] R. M. Entman, Framing: Toward clarification of a fractured paradigm, Journal of communication [67] A. Haleem, M. Javaid, R. P. Singh, An era of chatgpt as a significant futuristic support tool: A study on features, abilities, and challenges, BenchCouncil transactions on benchmarks, standards and evaluations 2 (2022) 100089. [68] N. Muralidhar, H. Rangwala, E.-H. S. Han, Recommending temporally relevant news content from implicit feedback data, in: 2015 IEEE 27th International Conference on Tools with Artificial Intelligence (ICTAI), IEEE, 2015, pp. 689–696. [69] A. D. Starke, S. Ø. Larsen, C. Trattner, Predicting feature-based similarity in the news domain using human judgments, in: Proceedings of the 9th international workshop on news recommendation and analytics (INRA 2021) co-located with 15th ACM conference on recommender systems (RecSys 2021), 2021. [70] M. Cinelli, G. De Francisci Morales, A. Galeazzi, W. Quattrociocchi, M. Starnini, The echo chamber efect on social media, Proceedings of the National Academy of Sciences 118 (2021) e2023301118. [71] E. R. Thompson, Development and validation of an internationally reliable short-form of the positive and negative afect schedule (panas), Journal of Cross-Cultural Psychology 38 (2007) 227–242. URL: https://doi.org/10.1177/0022022106297301. doi:10.1177/0022022106297301. arXiv:https://doi.org/10.1177/0022022106297301. [72] R. Fletcher, S. Park, The impact of trust in the news media on online news consumption and participation, Digital journalism 5 (2017) 1281–1299. [73] H. I. Chyi, Willingness to pay for online news: An empirical study on the viability of the subscription model, Journal of Media Economics 18 (2005) 131–142. URL: https://doi.org/10.1207/ s15327736me1802_4. doi:10.1207/s15327736me1802\_4. [74] B. P. Knijnenburg, M. C. Willemsen, Evaluating recommender systems with user experiments, in:

Recommender systems handbook, Springer, 2015, pp. 309–352. [75] D. DeSteno, R. E. Petty, D. D. Rucker, D. T. Wegener, J. Braverman, Discrete emotions and persuasion: the role of emotion-induced expectancies., Journal of personality and social psychology 86 (2004) 43. [76] R. Kühne, C. Schemer, The emotional efects of news frames on information processing and opinion formation, Communication Research 42 (2015) 387–407. [77] A. H. Eagly, S. Chaiken, The psychology of attitudes., Harcourt brace Jovanovich college publishers, 1993. [78] I. J. Roseman, C. Wiest, T. S. Swartz, Phenomenology, behaviors, and goals diferentiate discrete emotions., Journal of personality and social psychology 67 (1994) 206. [79] J. M. Sontag, Visual framing efects on emotion and mental health message efectiveness, Journal of Communication in Healthcare 11 (2018) 30–47. [80] S. Lecheler, L. Bos, R. Vliegenthart, The mediating role of emotions: News framing efects on opinions about immigration, Journalism & Mass Communication Quarterly 92 (2015) 812–838. [81] R. L. Nabi, C. M. Dobmeier, C. L. Robbins, D. Pérez Torres, N. Walter, Efects of scanning health news headlines on trust in science: an emotional framing perspective, Health Communication (2024) 1–13.

References

[1] M. Karimi , D. Jannach , M. Jugovac , News recommender systems-survey and roads ahead , Information Processing & Management 54 ( 2018 ) 1203 - 1227 .

[2] A. L. Opdahl , B. Tessem , D.-T. Dang-Nguyen, E. Motta , V. Setty , E. Throndsen , A. Tverberg , C. Trattner , Trustworthy journalism through ai , Data & Knowledge Engineering 146 ( 2023 ) 102182 .

[3] J. V. Pavlik , Collaborating with chatgpt: Considering the implications of generative artificial intelligence for journalism and media education , Journalism & Mass Communication Educator 78 ( 2023 ) 84 - 93 .

[4] M. Hansen , M. Roca-Sales , J. M. Keegan , G. King, Artificial intelligence: Practice and implications for journalism ( 2017 ).

[5] G. Romeo , E. Griglié, Ai ethics and policies: Why european journalism needs more of both , in: The 2021 Yearbook of the Digital Ethics Lab , Springer, 2022 , pp. 229 - 245 .

[6] BBC LLM Use Cases , Llm use cases at bbc, 2024 . URL: https://tech.ebu.ch/publications/ llm-use - cases - at-bbc, accessed: 2024 -08-12.

[7] Schibsted AI Strategy , Schibsted's ai strategy with sven størmer thaulow , 2024 . URL: https:// aiinside.show/episode/schibsteds-ai -strategy-with-sven-stormer-thaulow , accessed: 2024 -08-12.

[8] Reuters Institute for the Study of Journalism , Digital news report 2023: Executive summary , 2023 . URL: https://reutersinstitute.politics.ox.ac.uk/digital-news-report/2023/dnr-executive-summary, accessed: 2024 -08-21.

[9] N. Diakopoulos , Automating the news: How algorithms are rewriting the media , Harvard University Press, 2019 .

[10] S. Petridis , N. Diakopoulos , K. Crowston , M. Hansen , K. Henderson , S. Jastrzebski , J. V. Nickerson , L. B. Chilton , Anglekindling: Supporting journalistic angle ideation with large language models , in: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems , 2023 , pp. 1 - 16 .

[11] C. H. De Vreese , News framing: Theory and typology , Information design journal+ document design 13 ( 2005 ) 51 - 62 .

[12] T. Bolsen , M. A. Shapiro , The us news media, polarization on climate change, and pathways to efective communication , Environmental Communication 12 ( 2018 ) 149 - 163 .

[13] C. de Vreese , J. Peter, H. Semetko , et al., Framing politics at the launch of the euro: A cross-national comparative study of frames in the news ., Political Communication 18 ( 2001 ).

[14] P. López-Rabadán , Framing studies evolution in the social media era. digital advancement and reorientation of the research agenda , Social Sciences 11 ( 2021 ) 9 .

[15] R. Coleman , S. Banning , Network tv news' afective framing of the presidential candidates: Evidence for a second-level agenda-setting efect through visual framing , Journalism & Mass Communication Quarterly 83 ( 2006 ) 313 - 328 .

[16] K. E. McIntyre , R. Gibson , Positive news makes readers feel good: A “silver-lining” approach to negative news can attract audiences , Southern Communication Journal 81 ( 2016 ) 304 - 315 .

[17] J. Berger , K. L. Milkman , What makes online content viral? , Journal of marketing research 49 ( 2012 ) 192 - 205 . 43 ( 1993 ) 51 - 58 .

[42] S. T. Fiske , S. E. Taylor , Social cognition, Mcgraw-Hill Book Company, 1991 .

[43] D. Tewksbury , D. A. Scheufele , News framing theory and research , in: Media efects, Routledge, 2009 , pp. 33 - 49 .

[44] H. A. Semetko , P. M. Valkenburg , Framing european politics: A content analysis of press and television news , Journal of communication 50 ( 2000 ) 93 - 109 .

[45] J. B. Hester , R. Gibson , The economy and second-level agenda setting: A time-series analysis of economic news and public opinion about the economy , Journalism & Mass Communication Quarterly 80 ( 2003 ) 73 - 90 .

[46] E. Siapera , L. Papadopoulou, Hate as a 'hook': The political and afective economy of 'hate journalism' , Journalism 22 ( 2021 ) 1256 - 1272 .

[47] M. Maiese , R. Hanna , The mind-body politic , Springer, 2019 .

[48] J. Protevi , Covid -19 in the united states as afective frame , Frontiers in Psychology 13 ( 2022 ) 897215 .

[49] M. Sanford , M. Witkowska , R. Giford , M. Formanowicz , Emotional framing in online environmental activism: Pairing a twitter study with an ofline experiment , Frontiers in Psychology 13 ( 2023 ) 1099331 .

[50] T. Greenaway , K. S. Fielding , Positive afective framing of information reduces risk perceptions and increases acceptance of recycled water , Environmental Communication 14 ( 2020 ) 391 - 402 .

[51] M. Trussler , S. Soroka , Consumer demand for cynical and negative news frames , The International Journal of Press/Politics 19 ( 2014 ) 360 - 379 .

[52] K. Gross , L. D'ambrosio, Framing emotional response , Political psychology 25 ( 2004 ) 1 - 29 .

[53] A. D. Starke , K. Emami , A. Makarová , B. Ferwerda , Using visual and linguistic framing to support sustainable decisions in an online store ( 2023 ).

[54] C. Brantner , S. Geise , K. Lobinger , Fractured paradigm? theories, concepts and methodology of visual framing research: a systematic review , in: Annual Conference of the International Communication Association (ICA) -visual communication studies division , 2012 , pp. 1 - 40 .

[55] L. Rodriguez , D. V. Dimitrova , The levels of visual framing , Journal of visual literacy 30 ( 2011 ) 48 - 65 .

[56] R. Barthes , Image-music-text , volume 6135 , Macmillan , 1977 .

[57] C. J. Patrick , S. A. Lavoro , Ratings of emotional response to pictorial stimuli: Positive and negative afect dimensions , Motivation and Emotion 21 ( 1997 ) 297 - 321 .

[58] G. Salazar , M. C. Monroe , M. Ennes , J. A. Jones , D. Veríssimo , Testing the influence of visual framing on engagement and pro-environmental action , Conservation Science and Practice 4 ( 2022 ) e12812 .

[59] R. R. Mourão , D. K. Brown , Black lives matter coverage: How protest news frames and attitudinal change afect social media engagement , Digital Journalism 10 ( 2022 ) 626 - 646 .

[60] P. D'Angelo , Doing news framing analysis II: Empirical and theoretical perspectives , Routledge , 2018 .

[61] J. Bingaman , P. R. Brewer , A. Paintsil , D. C. Wilson, “ siri, show me scary images of ai”: Efects of text-based frames and visuals on support for artificial intelligence , Science Communication 43 ( 2021 ) 388 - 401 .

[62] M. E. Grabe , E. P. Bucy , Image bite politics: News and the visual framing of elections , Political Psychology, 2009 .

[63] M. C. Nisbet , Framing science: A new paradigm in public engagement , in: Communicating science, Routledge, 2009 , pp. 54 - 81 .

[64] A. R. Schuck , C. H. de Vreese , Reversed mobilization in referendum campaigns: How positive news framing can mobilize the skeptics , The International Journal of Press/Politics 14 ( 2009 ) 40 - 66 .

[65] N. Corbu , A. Bârgăoanu , F. Durach , G. Udrea, Fake news going viral: The mediating efect of negative emotions , Media Literacy and Academic Research 4 ( 2021 ) 58 - 87 .

[66] S. Salgado , G. Bobba, News on events and social media: A comparative analysis of facebook users' reactions , Journalism studies 20 ( 2019 ) 2258 - 2276 .