News shape public perception of events. The amount and frequency with which publishers release articles continues to increase. News recommender systems are tools designated to support readers in nding the most relevant articles. These systems struggle with challenging conditions. Readers refuse to express their interests explicitly, and publishers cannot reliably track them to infer their preferences. Publishers have to update their models to accommodate recent trends continuously. In this work, we aim to explore the use of contextual information to improve news recommender systems. We mine characteristic patterns and discuss how these ndings can help to develop innovative recommendation strategy and better evaluation protocols.
Publishers have created digital outlets to accommodate customers' demands for quicker access to recent news. The digital revolution has shifted the publication sector toward an \attention economy" [BO12]. Customers, who relinquish subscribing to a publisher, pay instead with their attention to advertisements. These customers appear to represent the majority as publishers struggle to compete and report declining revenues, especially in print. Consequently, publishers look for innovative ways to keep visitors interested.
Readers, on the other hand, have limited time available to engage with digital media. The amount of Copyright © CIKM 2018 for the individual papers by the papers' authors. Copyright © CIKM 2018 for the volume as a collection by its editors. This volume and its papers are published under the Creative Commons License Attribution 4.0 International (CC
These conditions disallow to directly apply standard algorithms such as collaborative ltering or contentbased ltering. Standard algorithms assume somewhat static collections of users and items with su ciently many interactions between both groups. As a result, content-based ltering and collaborative ltering struggle with the so-called cold-start problem. Therein, the recommender systems lack information about users' preferences toward items. This situation occurs for new users or items. Instead, publishers tend to focus on session-based recommendations, breaking news, and popularity-based recommender approaches. Session-based recommendations fail to capture readers' long-term interests. Breaking news and popularity-based methods relinquish personalization in favor of simplicity.
In this paper, we analyze readers' engagement with news to determine the importance of contextual information. Contextual information, such as the device type, time, and day, gives us clues about users' contextual news reading behavior. Based on the results from these analyses we argue that publishers should pay more attention to contextual aspects.
Our ndings support publishers to enhance their news recommendation systems. In particular, publishers can improve their evaluation practices to avoid suboptimal con gurations. Publishers monitor the activity on their websites by recording event-based logles. They tend to conduct experiments in the form of A/B tests [KL17]. In these tests, the system randomly assigns users to groups and presents each group a xed variation of the website. The websites may di er in style, layout, or content. We argue that publishers have to pay attention to contextual information to draw meaningful conclusions from their data. Speci cally, publishers risk suboptimal con gurations if user feedback correlates with contextual settings rather than the website.
This paper's remainder presents the following parts. Section 2 reviews existing approaches for news recommender systems. Section 3 dives deep into the contextual aspects of news engagement. Section 4 discusses the ndings of our data analysis concerning the di erent news publishers. Section 5 concludes and indicates further directions for research on context-aware news recommender systems. 2
Users' perception of items changes over time [SJNP12]. They may enjoy a particular item once but nd it less valuable later on. Alternatively, the may like some items in speci c circumstances. Context features describe circumstances in which users interact with items. Context-aware recommender systems use contextual data to provide more relevant recommendations. In [AT11], the authors discuss the context dimensions within a recommender system environment and how they change over time. Users' ongoing interactions with the system generate context information. Recommender systems use this information to enhance the instantaneous recommendations or user pro les. They include contextual information either via pre- ltering, post- ltering, or modeling. The approaches di er concerning the stage at which contextual information a ects the recommendations. Pre- ltering introduces a set of recommendation models each of which serves a particular context. Post- ltering takes the recommendations and excludes items according to the context. Context modeling integrates contextual information into the recommendation models. Context-aware recommendations have entered a variety of domains including movies [KABO10, LA13, LWW15, GRST10], music [BKL+11, WRW12], locations [YSC+13], tourism [ZBM12], and apps [SKB+12].
The news domain confronts context-aware recommender systems with particular challenges. Accordingly, types and usage of contextual features have to be customized [O GE14, KJJ18]. The news domain experiences more dynamics than other domains. For instance, publishers keep extending the set of available items by adding new articles. In addition, users' interests rapidly change, and news recommender systems struggle to anticipate these changes. In [JNT10], the authors explain that users' current context a ects their choice of news articles to read. Based on these ndings, the authors of [LZYL14] propose a model which extends the user pro le with contextual features. Recommender systems strive to consider both short-term and long-term reading interests. As a result, they build user pro les for both settings in an attempt to re ect shorter and more extended periods. Similarly, in [WZL+15], the authors present a hybrid context-aware news recommender system which uses explicit and implicit indicators (e.g. location, ratings, and reading time) to calculate the users' long-term and short-term interests. Herein, capturing long-term preferences represents a signi cant challenge as systems struggle to recognize reoccurring visitors. In [CC09], the authors propose a semantic runtime context complementing the recommendation algorithm. Thereby, they ease matching user and item pro les. In addition, they point out that widely used contextual features including recently engage items, computing platforms, network conditions, the social and physical environment, as well as the location a ect performance. Keeping the context information updated presents another challenge to news recommender systems. This is due to the domain's highly dynamic nature with publishers continuously releasing fresh content. In [GDF13], the authors propose a news recommender system based on context trees in order to establish a dynamically changing model. Context trees model sequences of news, topics, and topic distributions. They evolve following users' behavior and news trends. Context trees' structure facilitates responding quickly to incoming requests as paths usually involve only a few vertices. In [Lom14], the author introduces a context-aware ensemble for news recommendation. The ensemble blends di erent news recommendation algorithms. Simultaneously, the system records context-dependent performances. Subsequently, the ensemble can determine the best candidate algorithm for a given context. The paper documents what publishers can gain by considering contextual aspects compared to a popularitybased baseline. In [SSZ18], the authors consider a session as the vehicle of a contextual setting. They argue that readers' interest in news depends on their current situation and propose a session-based news recommender system. The system integrates both collaborative and content-based ltering to create shortterm interest models. In [SKP13], the authors consider users' location as important contextual factor. They argue that users reading news on their mobile devices are interested more in the news concerning their local surroundings. Consequently, news recommender systems ought to emphasize articles related to close-by events to match the context. In [LSC+10], the authors have access to a broader set of features describing users and articles. The considered features including users' location and articles' categories. Having reduced the feature space's dimensionality, a contextual bandit algorithm consumes the data. The contextual bandit selects recommendation models in a context-dependent fashion.
The consideration of contextual factors is paramount when evaluating recommender algorithms [BBL+16]. Changing contextual features renders experimental results irreproducible. Hence, evaluation protocols for news recommender systems must consider context to avoid falling for suboptimal con gurations.
Overall, the existing research shows that contextual features have a high in uence on the performance of recommender systems. There is still a need for a better understanding of contextual factors in news recommender systems. In particular, research has yet to determine which contextual features matter in which domain. 3
Understanding users' needs and habits is the key to provide suitable recommendations. We expect needs and habits to depend on the context such as the device type|desktop, mobile, or tablet|or the time and day. Users with limited time or browsing on mobile devices|with limited screen size to display articles| might prefer short breaking news over long, detailed stories. Some topics may frequently appear at certain times. Groups of readers with similar taste may appear more concentrated at times, for example, readers may like to read sports news at the weekend more than weekdays.
Other contextual features can be taken into account for a more complex user behavior analysis. However, we think that these three contextual features are the most prevalent ones to start such an analysis. They are also the most common contextual information available in existing datasets that we can use without violating users' privacy. We will analyze each of these aspects by looking at log les in order to verify this assumption. Publishers record user engagement on their platforms based on the interaction amid browser and web server. The resulting log les show events along with the context in which they occurred. We analyze four publishers' log les. Three publishers are located in Germany; one is situated in Norway, see Table 1. Three publishers blend regional and general news, whereas one focuses on car-related news. Our analysis considers two contextual dimensions. First, we look at the time at which events took place. Therein, we distinguish between the daytime|measured as the hour of the day|and the weekday. Second, we investigate the kind of device used to read news articles. We consider desktops, mobile devices, and tablets. In addition, we explore how categories a ect user engagement. We introduce the data underpinning our analysis. The log les come from the websites of four publishers referred to as A, B, C, and D. Publisher A's data have been recorded in the time from January 1 to March 31, 2017. The remaining publishers' data have been recorded in March 2017. Whenever a reader has loaded an article from any of the publishers' websites, the system has appended a line to the log le. As a result, we obtain a list of impressions for each of the publishers. Each impression entails a timestamp from which we derive the hour of the day as well as the weekday. Besides, each impression expresses which kind of device had been used to access the news article. Table 2 shows the number as well as the proportion of impressions for each device. Table 3 shows the number and proportion of impressions by weekday. 3.3
Users engage with websites in di erent ways. Publishers may look at what pages users decide to visit. We refer to this interaction as impression. Besides, users click on recommendations, dwell on pages, or write comments. We focus exclusively on impressions. We gauge users' engagement with publishers' services. Impressions occur in various contextual circumstances. We consider the choice of the device, the hour of the day, and weekday as dimensions.
Figure 5 encodes the user activity for various contexts in the form of a heat map. Each row consists of four heat maps linked to a particular kind of device. Each column refers to a particular publisher. The heat maps stretch the time dimensions hour of day and weekday. The colors encode the relative degree of activity in that particular con guration as explained by the legends on each heat map's right-hand side. In other words, the hotter the color, the more this particular context contributes to the overall activity.
All heat maps share one commonality. The nighttime throughout the week shows the lowest level of activity independent of the context or the publisher. The desktop usage tends to concentrate on usual working days. In contrast, at the weekends a smaller number of impressions originated from desktop devices. Tablets' activity appears primarily focused on the evenings with additional activity on the daytime on weekends. Mobile devices, such as smartphones, share this tendency. On the other hand, they spread additional activity throughout the daytime even on working days.
Figure 1 illustrates how publishers' weekend activity changes over the time of day. Again, the nighttime shows comparatively little activity. Still, in the afternoon the levels of activity show distinct patterns. 3.4
Publishers release news articles in a context-dependent fashion. Some stories allow journalists to prepare texts in advance. For instance, sport-related organizations, governmental bodies, and publicly traded companies announce schedules for competitions, votes, and shareholders' meetings. Other stories break unexpectedly such as natural catastrophes, tra c accidents, and celebrity deaths.
Figure 6 shows the times when publishers have added new articles. Two clocks refer to the hours of the day for three publishers. The number of new articles is color-coded according to the legend below. All publishers appear to add a large number of articles at night. This phenomenon is particularly striking for publisher D which appears to use the night exclusively to publish content. Publishers of car-related news generally deal with few breaking stories. The remaining publishers add new articles over the course of the day as well.
Figure 3 looks at the number of impressions for articles assigned to di erent categories. News concerning nance and health show a noticeable peak in the time between 1:00 p.m. and 2:00 p.m. Impressions appear to center around the noon with lesser peaks in the morning and afternoon. Figure 4 illustrates the number of articles published for di erent categories. Compared to the previous ndings, the publications spread more evenly over the working hours. We observe two signi cant peaks between 10:00 a.m. and 11:00 a.m. as well as between 1:00 p.m. and 2:00 p.m. Figure 2 contrasts the previous illustrations. The ordinate depicts the number of impressions. The abscissa shows the number of articles published. The various categories are color-coded according to the legend. Aside from each point, the gure presents the average number of impressions for each published article. We observe that articles related to nance and health accumulate many more impressions as the remaining categories. Still, the relatively few articles published about literature attract relatively more impressions. Articles related to cars attract the fewest readers. 4
Our analysis has shown that user engagement varies considerably with time, device, and publisher. Users hardly engage with news services at night. Working hours show users heavily using desktops in favor of tablets and mobile devices. Evenings and weekends tell a di erent story. Then, users engage more via mobile devices and tablets. We may expect di erent device usage over the course of the week and day due to readers' habits and lifestyles. Still, we need to pay attention to these di erences to optimize our news recommender systems. We have observed variations among the readers located in di erent countries. Norwegian readers start reading news earlier in the morning compared to readers in Germany. Conversely, German readers stop later in the evening their exploration of the news landscape. These di erences could be the result of culture and lifestyle. Furthermore, our analysis indicates that users' engagement di ers between publishers based on the subject and A B C
D
desktop mobile tablet
cars city of Cologne digital family finance health literature travel Publisher C 9 8 21 20 10 7 22 19 6 23 18 2000 5 12 17 4000 1 4 13 16 2 3 14 15 9 8 21 20 10 7 22 19 6 23 18 2000 5 12 17 4000 1 4 13 16 2 3 14 15 9 8 21 20 10 7 22 19 6 23 5 12 18 17 1 4 13 16 2 3 14 15 0
In addition, our analysis has shown that publishers release articles with varying schedules. Publishers can prepare stories about anticipated events or opinions. News recommender systems have to adapt to situations with radically changing item collections. On the one hand, a push of many prepared stories at night adds uncertainty to the story selection. On the other hand, breaking news stories attract a majority of attention. Publishers may rely heavily on A/B testing protocols to optimize their recommendation services. These evaluation tools monitor the behavior of disjoint groups of users each of which experiences a di erent system con guration. The longer the data gathering progresses, the more the contextual patterns fade. Hence, publishers optimize their systems for the most wide-spread contexts and abandon potential gains attainable by more ne-grained contextual analyses. Instead, our analysis suggests monitoring user behavior Publishers operate in dynamic, digital environments. They compete for the attention of users to monetize their content. News recommender systems facilitate readers' access to information. As a result, publishers continue to optimize their recommendations. Our
Conclusion and Future Work 5 analysis has shown that readers' engagement varies considerably in between contextual settings. We have considered the time and device as contextual dimensions. This kind of engagement re ects readers' daily routines. We have observed negligible activity at night, mostly desktop usage on working hours, and mobile device usage in the evenings and on weekends. Publishers release schedules vary considerably. Prepared articles emerge in the night while stories related to breaking events enter irregularly. The example of publisher D shows that news categories di er in popularity. Besides, the categories' popularity changes over the course of the day.
We see multiple directions to extend this line of thought. Publishers ought to devise methods to introduce context-awareness to their systems. Conducting user studies yields clearer insight into readers' requirements. Alternatively, publishers could extend A/B testing to encompass longer periods and simultaneously monitor contextual features. Further analysis of contextual reader engagement could include additional publishers to verify the discussed results. Establishing publisher-speci c recommendations demands a more in-depth analysis of contextual information and readers' behavior. [AT11] [BO12]
Erik Brynjolfsson and JooHee Oh. The
attention economy: Measuring the value of
free digital services on the internet. In
ICIS. Association for
Ivan Cantador and Pablo Castells. Semantic contextualisation in a news recommender system. Workshop on ContextAware Recommender Systems, 2009.
Florent Garcin, Christos Dimitrakakis, and Boi Faltings. Personalized news recommendation with context trees. In RecSys, pages 105{112, New York, 2013. ACM Press. [GRST10] Zeno Gantner, Ste en Rendle, and Lars Schmidt-Thieme. Factorization models for context-/time-aware movie recommendations. In Proceedings of the Workshop on Context-Aware Movie Recommendation, pages 14{19. ACM, 2010. [GZL+17] Jon Atle Gulla, Lemei Zhang, Peng Liu, O zlem O zgobek, and Xiaomeng Su. The adressa dataset for news recommendation.
In Proceedings of the International Conference on Web Intelligence, pages 1042{ 1048. ACM, 2017. [JNT10]
Jeremy Jancsary, Friedrich Neubarth, and Harald Trost. Towards context-aware personalization and a broad perspective on the semantics of news articles. In Proceedings of the Fourth ACM Conference on Recommender Systems, RecSys '10, pages 289{ 292, New York, 2010. ACM. [KABO10] Alexandros Karatzoglou, Xavier Amatriain, Linas Baltrunas, and Nuria Oliver. Multiverse recommendation: n-dimensional tensor factorization for context-aware collaborative ltering. In Proceedings of the fourth ACM conference on Recommender systems, pages 79{86.
ACM, 2010. [KHBH13] Benjamin Kille, Frank Hopfgartner, Torben Brodt, and Tobias Heintz. The plista dataset. In Proceedings of the 2013 International News Recommender Systems Workshop and Challenge, pages 16{23.
ACM, 2013. [KJJ18] [KL17]
Mozhgan Karimi, Dietmar Jannach, and Michael Jugovac. News recommender systems - Survey and roads ahead. Information Processing and Management, 2018.
Ron Kohavi and Roger Longbotham. Online controlled experiments and a/b testing. In Encyclopedia of Machine Learning and Data Mining, pages 922{929. Springer, 2017. [O GE14]