Defining Contextual Factors for News Consumption Benjamin Kille Özlem Özgöbek Technische Universität Berlin Norwegian University of Science and Technology benjamin.kille@tu-berlin.de ozlem.ozgobek@ntnu.no Andreas Lommatzsch Technische Universität Berlin andreas.lommatzsch@dai-labor.de published content exceeds their attention capacity by far. This incites intense competition among publish- Abstract ers in which publishers seek to direct the most relevant content to readers. They reason that readers will be- News shape public perception of events. The come loyal to services providing their desired contents. amount and frequency with which publishers Advertisers, on the other hand, demand evidence for release articles continues to increase. News users’ attention in order to measure the spread of their recommender systems are tools designated to information. Consequently, publishers need to moni- support readers in finding the most relevant tor their platforms to provide the requested evidence. articles. These systems struggle with chal- Publishers have started to employ news recom- lenging conditions. Readers refuse to express mender systems to gain competitive advantages. Rec- their interests explicitly, and publishers can- ommender systems assist readers in navigating more not reliably track them to infer their prefer- conveniently by suggesting a subset of articles. Ide- ences. Publishers have to update their models ally, the subset matches readers’ preferences. Thereby, to accommodate recent trends continuously. readers can avoid searching for exciting content them- In this work, we aim to explore the use of con- selves and save time. textual information to improve news recom- mender systems. We mine characteristic pat- The news domain meets existing recommendation terns and discuss how these findings can help algorithms with challenging conditions. Many read- to develop innovative recommendation strat- ers visit publishers’ websites without registering a user egy and better evaluation protocols. profile. Consequently, publishers struggle to link read- ers’ sessions. They fail to establish expressive user 1 Introduction profiles. Further, articles tend to remain relevant for merely a short period. News stories tend to relate to Publishers have created digital outlets to accommo- events which publishers cannot anticipate in advance. date customers’ demands for quicker access to recent These conditions disallow to directly apply standard news. The digital revolution has shifted the publi- algorithms such as collaborative filtering or content- cation sector toward an “attention economy” [BO12]. based filtering. Standard algorithms assume some- Customers, who relinquish subscribing to a publisher, what static collections of users and items with suffi- pay instead with their attention to advertisements. ciently many interactions between both groups. As These customers appear to represent the majority as a result, content-based filtering and collaborative fil- publishers struggle to compete and report declining tering struggle with the so-called cold-start problem. revenues, especially in print. Consequently, publishers Therein, the recommender systems lack information look for innovative ways to keep visitors interested. about users’ preferences toward items. This situa- Readers, on the other hand, have limited time avail- tion occurs for new users or items. Instead, publish- able to engage with digital media. The amount of ers tend to focus on session-based recommendations, Copyright © CIKM 2018 for the individual papers by the papers' breaking news, and popularity-based recommender ap- authors. Copyright © CIKM 2018 for the volume as a collection proaches. Session-based recommendations fail to cap- by its editors. This volume and its papers are published under ture readers’ long-term interests. Breaking news and the Creative Commons License Attribution 4.0 International (CC BY 4.0). popularity-based methods relinquish personalization tions and excludes items according to the context. in favor of simplicity. Context modeling integrates contextual information In this paper, we analyze readers’ engagement with into the recommendation models. Context-aware rec- news to determine the importance of contextual infor- ommendations have entered a variety of domains in- mation. Contextual information, such as the device cluding movies [KABO10, LA13, LWW15, GRST10], type, time, and day, gives us clues about users’ con- music [BKL+ 11, WRW12], locations [YSC+ 13], textual news reading behavior. Based on the results tourism [ZBM12], and apps [SKB+ 12]. from these analyses we argue that publishers should The news domain confronts context-aware recom- pay more attention to contextual aspects. mender systems with particular challenges. Accord- Our findings support publishers to enhance their ingly, types and usage of contextual features have to news recommendation systems. In particular, pub- be customized [ÖGE14, KJJ18]. The news domain lishers can improve their evaluation practices to avoid experiences more dynamics than other domains. For suboptimal configurations. Publishers monitor the ac- instance, publishers keep extending the set of avail- tivity on their websites by recording event-based log- able items by adding new articles. In addition, users’ files. They tend to conduct experiments in the form interests rapidly change, and news recommender sys- of A/B tests [KL17]. In these tests, the system ran- tems struggle to anticipate these changes. In [JNT10], domly assigns users to groups and presents each group the authors explain that users’ current context affects a fixed variation of the website. The websites may their choice of news articles to read. Based on these differ in style, layout, or content. We argue that pub- findings, the authors of [LZYL14] propose a model lishers have to pay attention to contextual informa- which extends the user profile with contextual fea- tion to draw meaningful conclusions from their data. tures. Recommender systems strive to consider both Specifically, publishers risk suboptimal configurations short-term and long-term reading interests. As a re- if user feedback correlates with contextual settings sult, they build user profiles for both settings in an rather than the website. attempt to reflect shorter and more extended periods. This paper’s remainder presents the following parts. Similarly, in [WZL+ 15], the authors present a hybrid Section 2 reviews existing approaches for news recom- context-aware news recommender system which uses mender systems. Section 3 dives deep into the contex- explicit and implicit indicators (e.g. location, ratings, tual aspects of news engagement. Section 4 discusses and reading time) to calculate the users’ long-term and the findings of our data analysis concerning the differ- short-term interests. Herein, capturing long-term pref- ent news publishers. Section 5 concludes and indicates erences represents a significant challenge as systems further directions for research on context-aware news struggle to recognize reoccurring visitors. In [CC09], recommender systems. the authors propose a semantic runtime context com- plementing the recommendation algorithm. Thereby, they ease matching user and item profiles. In addition, 2 Related Work they point out that widely used contextual features Users’ perception of items changes over time [SJNP12]. including recently engage items, computing platforms, They may enjoy a particular item once but find it network conditions, the social and physical environ- less valuable later on. Alternatively, the may like ment, as well as the location affect performance. Keep- some items in specific circumstances. Context fea- ing the context information updated presents another tures describe circumstances in which users interact challenge to news recommender systems. This is due with items. Context-aware recommender systems use to the domain’s highly dynamic nature with publishers contextual data to provide more relevant recommen- continuously releasing fresh content. In [GDF13], the dations. In [AT11], the authors discuss the context authors propose a news recommender system based dimensions within a recommender system environ- on context trees in order to establish a dynamically ment and how they change over time. Users’ ongo- changing model. Context trees model sequences of ing interactions with the system generate context in- news, topics, and topic distributions. They evolve fol- formation. Recommender systems use this informa- lowing users’ behavior and news trends. Context trees’ tion to enhance the instantaneous recommendations structure facilitates responding quickly to incoming re- or user profiles. They include contextual informa- quests as paths usually involve only a few vertices. tion either via pre-filtering, post-filtering, or mod- In [Lom14], the author introduces a context-aware eling. The approaches differ concerning the stage ensemble for news recommendation. The ensemble at which contextual information affects the recom- blends different news recommendation algorithms. Si- mendations. Pre-filtering introduces a set of rec- multaneously, the system records context-dependent ommendation models each of which serves a partic- performances. Subsequently, the ensemble can deter- ular context. Post-filtering takes the recommenda- mine the best candidate algorithm for a given context. The paper documents what publishers can gain by con- 0.07 sidering contextual aspects compared to a popularity- Relative Proportion of Impressions on Weekends based baseline. In [SSZ18], the authors consider a ses- 0.06 sion as the vehicle of a contextual setting. They argue that readers’ interest in news depends on their cur- 0.05 rent situation and propose a session-based news rec- 0.04 ommender system. The system integrates both col- laborative and content-based filtering to create short- 0.03 term interest models. In [SKP13], the authors consider users’ location as important contextual factor. They 0.02 Publisher A argue that users reading news on their mobile devices Publisher B 0.01 Publisher C are interested more in the news concerning their local Publisher D surroundings. Consequently, news recommender sys- 0 tems ought to emphasize articles related to close-by 0 2 4 6 8 10 12 14 16 18 20 22 24 Hour of Day events to match the context. In [LSC+ 10], the authors have access to a broader set of features describing users Figure 1: Comparison of the relative proportions of and articles. The considered features including users’ impressions by publishers over the hour of day at week- location and articles’ categories. Having reduced the ends. Each curve refers to the relative proportion of feature space’s dimensionality, a contextual bandit al- impressions for a publisher, only considering impres- gorithm consumes the data. The contextual bandit se- sions occurring on Saturdays and Sundays. lects recommendation models in a context-dependent fashion. also the most common contextual information avail- The consideration of contextual factors is able in existing datasets that we can use without vi- paramount when evaluating recommender algo- olating users’ privacy. We will analyze each of these rithms [BBL+ 16]. Changing contextual features aspects by looking at log files in order to verify this renders experimental results irreproducible. Hence, assumption. evaluation protocols for news recommender systems must consider context to avoid falling for suboptimal 3.1 Approach configurations. Publishers record user engagement on their platforms Overall, the existing research shows that contextual based on the interaction amid browser and web server. features have a high influence on the performance of The resulting log files show events along with the con- recommender systems. There is still a need for a bet- text in which they occurred. We analyze four pub- ter understanding of contextual factors in news rec- lishers’ log files. Three publishers are located in Ger- ommender systems. In particular, research has yet to many; one is situated in Norway, see Table 1. Three determine which contextual features matter in which publishers blend regional and general news, whereas domain. one focuses on car-related news. Our analysis consid- ers two contextual dimensions. First, we look at the 3 Data Analysis time at which events took place. Therein, we distin- guish between the daytime—measured as the hour of Understanding users’ needs and habits is the key to the day—and the weekday. Second, we investigate the provide suitable recommendations. We expect needs kind of device used to read news articles. We consider and habits to depend on the context such as the de- desktops, mobile devices, and tablets. In addition, we vice type—desktop, mobile, or tablet—or the time and explore how categories affect user engagement. day. Users with limited time or browsing on mobile devices—with limited screen size to display articles— 3.2 Data Description might prefer short breaking news over long, detailed stories. Some topics may frequently appear at certain We introduce the data underpinning our analysis. The times. Groups of readers with similar taste may ap- log files come from the websites of four publishers re- pear more concentrated at times, for example, readers ferred to as A, B, C, and D. Publisher A’s data have may like to read sports news at the weekend more than been recorded in the time from January 1 to March weekdays. 31, 2017. The remaining publishers’ data have been Other contextual features can be taken into account recorded in March 2017. Whenever a reader has loaded for a more complex user behavior analysis. However, an article from any of the publishers’ websites, the sys- we think that these three contextual features are the tem has appended a line to the log file. As a result, most prevalent ones to start such an analysis. They are we obtain a list of impressions for each of the publish- ers. Each impression entails a timestamp from which Figure 6 shows the times when publishers have we derive the hour of the day as well as the weekday. added new articles. Two clocks refer to the hours of Besides, each impression expresses which kind of de- the day for three publishers. The number of new arti- vice had been used to access the news article. Table 2 cles is color-coded according to the legend below. All shows the number as well as the proportion of impres- publishers appear to add a large number of articles sions for each device. Table 3 shows the number and at night. This phenomenon is particularly striking for proportion of impressions by weekday. publisher D which appears to use the night exclusively to publish content. Publishers of car-related news gen- 3.3 User Engagement erally deal with few breaking stories. The remaining publishers add new articles over the course of the day Users engage with websites in different ways. Pub- as well. lishers may look at what pages users decide to visit. Figure 3 looks at the number of impressions for arti- We refer to this interaction as impression. Besides, cles assigned to different categories. News concerning users click on recommendations, dwell on pages, or finance and health show a noticeable peak in the time write comments. We focus exclusively on impressions. between 1:00 p.m. and 2:00 p.m. Impressions appear We gauge users’ engagement with publishers’ services. to center around the noon with lesser peaks in the Impressions occur in various contextual circumstances. morning and afternoon. Figure 4 illustrates the num- We consider the choice of the device, the hour of the ber of articles published for different categories. Com- day, and weekday as dimensions. pared to the previous findings, the publications spread Figure 5 encodes the user activity for various con- more evenly over the working hours. We observe two texts in the form of a heat map. Each row consists significant peaks between 10:00 a.m. and 11:00 a.m. of four heat maps linked to a particular kind of de- as well as between 1:00 p.m. and 2:00 p.m. Figure 2 vice. Each column refers to a particular publisher. contrasts the previous illustrations. The ordinate de- The heat maps stretch the time dimensions hour of day picts the number of impressions. The abscissa shows and weekday. The colors encode the relative degree of the number of articles published. The various cate- activity in that particular configuration as explained gories are color-coded according to the legend. Aside by the legends on each heat map’s right-hand side. In from each point, the figure presents the average num- other words, the hotter the color, the more this par- ber of impressions for each published article. We ob- ticular context contributes to the overall activity. serve that articles related to finance and health accu- All heat maps share one commonality. The night- mulate many more impressions as the remaining cate- time throughout the week shows the lowest level of ac- gories. Still, the relatively few articles published about tivity independent of the context or the publisher. The literature attract relatively more impressions. Articles desktop usage tends to concentrate on usual working related to cars attract the fewest readers. days. In contrast, at the weekends a smaller number of impressions originated from desktop devices. Tablets’ 4 Discussion activity appears primarily focused on the evenings with additional activity on the daytime on weekends. Our analysis has shown that user engagement varies Mobile devices, such as smartphones, share this ten- considerably with time, device, and publisher. Users dency. On the other hand, they spread additional ac- hardly engage with news services at night. Working tivity throughout the daytime even on working days. hours show users heavily using desktops in favor of Figure 1 illustrates how publishers’ weekend activ- tablets and mobile devices. Evenings and weekends ity changes over the time of day. Again, the nighttime tell a different story. Then, users engage more via shows comparatively little activity. Still, in the after- mobile devices and tablets. We may expect differ- noon the levels of activity show distinct patterns. ent device usage over the course of the week and day due to readers’ habits and lifestyles. Still, we need 3.4 Publisher Activity to pay attention to these differences to optimize our news recommender systems. We have observed vari- Publishers release news articles in a context-dependent ations among the readers located in different coun- fashion. Some stories allow journalists to prepare texts tries. Norwegian readers start reading news earlier in in advance. For instance, sport-related organizations, the morning compared to readers in Germany. Con- governmental bodies, and publicly traded companies versely, German readers stop later in the evening their announce schedules for competitions, votes, and share- exploration of the news landscape. These differences holders’ meetings. Other stories break unexpectedly could be the result of culture and lifestyle. Further- such as natural catastrophes, traffic accidents, and more, our analysis indicates that users’ engagement celebrity deaths. differs between publishers based on the subject and Table 1: Publisher Description. The domain refers to the publishers’ spectrum of topics. The location refers to the publishers’ headquarters. The source refers to the data set’s origin. The duration refers to the length of the period during which the data have been collected. Publisher Domain Location Source Duration + A general & local Trondheim, Norway Adressa [GZL 17] three months B general & local Berlin, Germany NewsREEL [KHBH13] one month C general & local Cologne, Germany NewsREEL [KHBH13] one month D cars Berlin, Germany NewsREEL [KHBH13] one month Table 2: Number and proportion of impressions by publisher and device. Publishers are listed as columns, whereas rows refer to devices. The bottom row shows the total number of impressions per publisher. Publisher A Publisher B Publisher C Publisher D Device νimpressions % νimpressions % νimpressions % νimpressions % desktop 38 403 480 33.8 5 230 237 34.6 4 770 134 76.3 9 349 859 41.4 mobile 53 906 527 47.5 8 293 081 54.9 139 878 2.2 10 603 232 46.9 tablet 21 269 688 18.7 1 590 700 10.5 1 339 766 21.4 2 640 384 11.7 Σ 113 579 695 15 114 018 6 249 778 22 593 475 Table 3: Number and proportion of impressions by publisher and weekday. Publishers are listed as columns, whereas rows refer to weekdays. The bottom row shows the total number of impressions per publisher. Publisher A Publisher B Publisher C Publisher D Weekday νimpressions % νimpressions % νimpressions % νimpressions % Monday 17 326 061 15.3 1 968 745 13.0 954 698 15.3 3 141 987 13.9 Tuesday 17 574 750 15.5 2 322 038 15.4 958 494 15.3 3 041 217 13.5 Wednesday 17 664 762 15.6 2 511 516 16.6 1 146 296 18.3 3 684 470 16.3 Thursday 17 076 853 15.0 2 576 185 17.0 1 121 812 17.9 3 525 732 15.6 Friday 16 128 126 14.2 2 367 162 15.7 975 581 15.6 3 421 870 15.1 Saturday 12 522 288 11.0 1 627 104 10.8 521 625 8.3 2 762 025 12.2 Sunday 15 286 855 13.5 1 741 268 11.5 571 271 9.1 3 016 174 13.5 Σ 113 579 695 15 114 018 6 249 778 22 593 475 Publisher C cars 782 668 4.000.000 city of Cologne digital number of impressions family finance 3.000.000 health literature travel 2.000.000 921 ∅ imp / article 679 1.000.000 821 1338 92 0 421 0 2000 4000 6000 number of published articles Figure 2: Relation of number of articles published and number of impressions for the categories of publisher C Publisher C 4.000.000 cars city of Cologne number of impressions digital 3.000.000 family finance health 2.000.000 literature travel 1.000.000 0 0 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Figure 3: Number of impressions by category for publisher C. Publisher C 3.500 cars number of articles published 3.000 city of Cologne digital 2.500 family finance 2.000 health literature travel 1.500 1.000 500 0 0 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Figure 4: Number of articles published by category for publisher C. Publisher A | desktop Publisher C | desktop Publisher B | desktop Publisher D | desktop 23:00 νrel Impressions 23:00 νrel Impressions 23:00 νrel Impressions 23:00 νrel Impressions 22:00 [0,1) 22:00 [0,1) 22:00 [0,1) 22:00 [0,1) 21:00 21:00 [1,2) 21:00 [1,2) 21:00 [1,2) [1,2) [2,3) [2,3) [2,3) 20:00 [2,3) 20:00 20:00 20:00 [3,4) [3,4) [3,4) 19:00 [3,4) 19:00 [4,5) 19:00 [4,5) 19:00 [4,5) 18:00 [4,5) 18:00 [5,6) 18:00 [5,6) 18:00 [5,6) 17:00 [5,6) 17:00 [6,7) 17:00 [6,7) 17:00 [6,7) 16:00 [6,7) 16:00 [7,8) 16:00 [7,8) 16:00 [7,8) 15:00 15:00 [8,9) 15:00 [8,9) 15:00 [8,9) [7,8) [9,10) [9,10) [9,10) 14:00 [8,9) 14:00 14:00 14:00 [10,11) [10,11) [10,11) 13:00 [9,10) 13:00 [11,12) 13:00 [11,12) 13:00 [11,12) 12:00 12:00 [12,13) 12:00 [12,13) 12:00 [12,13) [10,11) 11:00 11:00 [13,14) 11:00 [13,14) 11:00 [13,14) [11,12) 10:00 10:00 [14,15) 10:00 [14,15) 10:00 [14,15) [12,13) [15,16) [15,16) [15,16) 9:00 9:00 9:00 9:00 [13,14) [16,17) ×10-3 ×10-3 8:00 8:00 8:00 8:00 [14,15) [17,18) 7:00 -3 7:00 7:00 7:00 6:00 ×10 6:00 [18,19) 6:00 N = 5,230,237 6:00 N = 9,349,859 [19,20) 5:00 5:00 ×10-3 5:00 5:00 4:00 N=38,403,480 4:00 4:00 4:00 3:00 3:00 N = 4,770,134 3:00 3:00 2:00 2:00 2:00 2:00 1:00 1:00 1:00 1:00 0:00 0:00 0:00 0:00 Mon Tue Wed Thu Fri Sat Sun Mon Tue Wed Thu Fri Sat Sun Mon Tue Wed Thu Fri Sat Sun Mon Tue Wed Thu Fri Sat Sun Publisher A | mobile Publisher C | mobile Publisher B | mobile Publisher D | mobile 23:00 νrel Impressions 23:00 νrel Impressions 23:00 νrel Impressions 23:00 νrel Impressions 22:00 [0,1) 22:00 [0,1) 22:00 [0,1) 22:00 [0,1) 21:00 [1,2) 21:00 [1,2) 21:00 [1,2) 21:00 [1,2) [2,3) [2,3) [2,3) [2,3) 20:00 20:00 20:00 20:00 [3,4) [3,4) [3,4) [3,4) 19:00 [4,5) 19:00 [4,5) 19:00 [4,5) 19:00 [4,5) 18:00 [5,6) 18:00 [5,6) 18:00 [5,6) 18:00 [5,6) 17:00 [6,7) 17:00 [6,7) 17:00 [6,7) 17:00 [6,7) 16:00 [7,8) 16:00 [7,8) 16:00 [7,8) 16:00 [7,8) 15:00 [8,9) 15:00 [8,9) 15:00 [8,9) 15:00 [8,9) 14:00 [9,10) 14:00 [9,10) 14:00 [9,10) 14:00 [9,10) [10,11) [10,11) [10,11) [10,11) 13:00 ×10-3 13:00 [11,12) 13:00 [11,12) 13:00 [11,12) 12:00 12:00 [12,13) 12:00 [12,13) 12:00 [12,13) 11:00 N = 53,906,527 11:00 [13,14) 11:00 [13,14) 11:00 [13,14) 10:00 10:00 [14,15) 10:00 [14,15) 10:00 [14,15) 9:00 9:00 [15,16) 9:00 [15,16) 9:00 [15,16) 8:00 8:00 ×10-3 8:00 ×10-3 8:00 ×10-3 7:00 7:00 7:00 7:00 N = 139,878 N = 8,293,081 N = 10,603,232 6:00 6:00 6:00 6:00 5:00 5:00 5:00 5:00 4:00 4:00 4:00 4:00 3:00 3:00 3:00 3:00 2:00 2:00 2:00 2:00 1:00 1:00 1:00 1:00 0:00 0:00 0:00 0:00 Mon Tue Wed Thu Fri Sat Sun Mon Tue Wed Thu Fri Sat Sun Mon Tue Wed Thu Fri Sat Sun Mon Tue Wed Thu Fri Sat Sun Publisher A | tablet Publisher C | tablet Publisher B | tablet Publisher D | tablet 23:00 νrel Impressions 23:00 νrel Impressions 23:00 νrel Impressions 23:00 νrel Impressions 22:00 [0,1) 22:00 [0,1) 22:00 [0,1) 22:00 [0,1) 21:00 [1,2) 21:00 [1,2) 21:00 [1,2) 21:00 [1,2) [2,3) [2,3) [2,3) [2,3) 20:00 20:00 20:00 20:00 [3,4) [3,4) [3,4) [3,4) 19:00 [4,5) 19:00 [4,5) 19:00 [4,5) 19:00 [4,5) 18:00 [5,6) 18:00 [5,6) 18:00 [5,6) 18:00 [5,6) 17:00 [6,7) 17:00 [6,7) 17:00 [6,7) 17:00 [6,7) 16:00 [7,8) 16:00 [7,8) 16:00 [7,8) 16:00 [7,8) 15:00 [8,9) 15:00 [8,9) 15:00 [8,9) 15:00 [8,9) 14:00 [9,10) 14:00 [9,10) 14:00 [9,10) 14:00 [9,10) [10,11) [10,11) [10,11) [10,11) 13:00 [11,12) 13:00 [11,12) 13:00 [11,12) 13:00 [11,12) 12:00 [12,13) 12:00 [12,13) 12:00 [12,13) 12:00 [12,13) 11:00 [13,14) 11:00 [13,14) 11:00 [13,14) 11:00 [13,14) 10:00 [14,15) 10:00 [14,15) 10:00 [14,15) 10:00 [14,15) 9:00 [15,16) 9:00 [15,16) 9:00 [15,16) 9:00 [15,16) 8:00 ×10-3 8:00 ×10-3 8:00 [16,17) 8:00 ×10-3 [17,18) 7:00 7:00 7:00 7:00 N = 21,269,688 N = 1,339,766 [18,19) N = 2,640,384 6:00 6:00 6:00 [19,20) 6:00 5:00 5:00 5:00 ×10-3 5:00 4:00 4:00 4:00 4:00 3:00 3:00 3:00 N = 1,590,700 3:00 2:00 2:00 2:00 2:00 1:00 1:00 1:00 1:00 0:00 0:00 0:00 0:00 Mon Tue Wed Thu Fri Sat Sun Mon Tue Wed Thu Fri Sat Sun Mon Tue Wed Thu Fri Sat Sun Mon Tue Wed Thu Fri Sat Sun Figure 5: Heat maps showing user activity by device, daytime, and weekday. Each row contains four heat maps related to a specific device. Each column contains three heat maps related to a particular publisher. The number of impressions is color-coded according to the legends on each heat maps right-hand side. The number of impressions in total for the context, or the pair of publisher and device, is shown below the legend. Publisher D Publisher B Publisher C 11 0 11 0 11 0 10 1 10 1 10 1 9 2 9 2 9 2 8 3 8 3 8 3 7 4 7 4 7 4 6 5 6 5 6 5 23 12 23 12 23 12 22 13 22 13 22 13 21 14 21 14 21 14 20 15 20 15 20 15 19 16 19 16 19 16 18 17 18 17 18 17 0 2000 4000 6000 0 500 1000 1500 0 2000 4000 6000 Figure 6: Illustration of the daytime when publishers release articles. For three publishers, each column presents two clocks. The clocks color-code the proportion of articles released over the course of half a day. type. Topic-specific publishers, such as publisher D, and determining groups with similar preferences. This exhibit markedly different engagement patterns com- opens up the opportunity to introduce a low-level per- pared to general news outlets. As a result, publishers sonalization which also respects users’ anonymity and need to pay close attention to their reader base and privacy. Moreover, publishers seek to maximize long- adapt to their particular needs. Especially, publishers term business goals. Some publishers have managed to must guarantee that new stories become available be- establish a large enough group of subscribed users to fore the majority of readers engages the service. Trans- cover their costs. For them, adding subscribers takes ferring knowledge from different domains represents a precedence over short-term user engagement. Pub- challenge. We cannot reliably track users engaging lishers need to align their recommendation strategies with different publishers. Hence, we struggle to estab- to their business goals. Consequently, recommenda- lish links between articles which suitably complement tion algorithms face different challenges depending on one another. the context. Tuning the recommendation algorithm to In addition, our analysis has shown that publishers users’ needs promises to help in achieving long-term release articles with varying schedules. Publishers can goals. Presenting shorter articles on mobile devices prepare stories about anticipated events or opinions. and topics popular in particular contexts, represent News recommender systems have to adapt to situa- two directions publishers could turn. Additionally, tions with radically changing item collections. On the publishers ought to consider contextual differences as one hand, a push of many prepared stories at night they internationalize their services. Our analysis in- adds uncertainty to the story selection. On the other dicates that culture and lifestyle affect readers’ per- hand, breaking news stories attract a majority of at- ception of recommendations. Consequently, publish- tention. Publishers may rely heavily on A/B testing ers need to take into account their recommendations’ protocols to optimize their recommendation services. destination. These evaluation tools monitor the behavior of dis- joint groups of users each of which experiences a differ- 5 Conclusion and Future Work ent system configuration. The longer the data gather- ing progresses, the more the contextual patterns fade. Publishers operate in dynamic, digital environments. Hence, publishers optimize their systems for the most They compete for the attention of users to monetize wide-spread contexts and abandon potential gains at- their content. News recommender systems facilitate tainable by more fine-grained contextual analyses. In- readers’ access to information. As a result, publish- stead, our analysis suggests monitoring user behavior ers continue to optimize their recommendations. Our analysis has shown that readers’ engagement varies [CC09] Iván Cantador and Pablo Castells. Se- considerably in between contextual settings. We have mantic contextualisation in a news recom- considered the time and device as contextual dimen- mender system. Workshop on Context- sions. This kind of engagement reflects readers’ daily Aware Recommender Systems, 2009. routines. We have observed negligible activity at night, mostly desktop usage on working hours, and [GDF13] Florent Garcin, Christos Dimitrakakis, and mobile device usage in the evenings and on weekends. Boi Faltings. Personalized news recommen- Publishers release schedules vary considerably. Pre- dation with context trees. In RecSys, pages pared articles emerge in the night while stories related 105–112, New York, 2013. ACM Press. to breaking events enter irregularly. The example of [GRST10] Zeno Gantner, Steffen Rendle, and Lars publisher D shows that news categories differ in popu- Schmidt-Thieme. Factorization models larity. Besides, the categories’ popularity changes over for context-/time-aware movie recommen- the course of the day. dations. In Proceedings of the Work- We see multiple directions to extend this line of shop on Context-Aware Movie Recommen- thought. Publishers ought to devise methods to intro- dation, pages 14–19. ACM, 2010. duce context-awareness to their systems. Conducting user studies yields clearer insight into readers’ require- [GZL+ 17] Jon Atle Gulla, Lemei Zhang, Peng Liu, ments. Alternatively, publishers could extend A/B Özlem Özgöbek, and Xiaomeng Su. The testing to encompass longer periods and simultane- adressa dataset for news recommendation. ously monitor contextual features. Further analysis of In Proceedings of the International Con- contextual reader engagement could include additional ference on Web Intelligence, pages 1042– publishers to verify the discussed results. Establishing 1048. ACM, 2017. publisher-specific recommendations demands a more [JNT10] Jeremy Jancsary, Friedrich Neubarth, and in-depth analysis of contextual information and read- Harald Trost. Towards context-aware per- ers’ behavior. sonalization and a broad perspective on the semantics of news articles. In Proceedings References of the Fourth ACM Conference on Recom- [AT11] Gediminas Adomavicius and Alexander mender Systems, RecSys ’10, pages 289– Tuzhilin. Context-aware recommender sys- 292, New York, 2010. ACM. tems. In Francesco Ricci, Lior Rokach, [KABO10] Alexandros Karatzoglou, Xavier Am- Bracha Shapira, and Paul B. Kantor, ed- atriain, Linas Baltrunas, and Nuria itors, Recommender Systems Handbook, Oliver. Multiverse recommendation: pages 217–253. Springer, 2011. n-dimensional tensor factorization for [BBL+ 16] Jöran Beel, Corinna Breitinger, Ste- context-aware collaborative filtering. In fan Langer, Andreas Lommatzsch, and Proceedings of the fourth ACM conference Bela Gipp. Towards reproducibility in on Recommender systems, pages 79–86. recommender-systems research. User ACM, 2010. Model. User-Adapt. Interact., 26(1):69– [KHBH13] Benjamin Kille, Frank Hopfgartner, Tor- 101, 2016. ben Brodt, and Tobias Heintz. The plista dataset. In Proceedings of the 2013 In- [BKL+ 11] Linas Baltrunas, Marius Kaminskas, ternational News Recommender Systems Bernd Ludwig, Omar Moling, Francesco Workshop and Challenge, pages 16–23. Ricci, Aykan Aydin, Karl-Heinz Lüke, and ACM, 2013. Roland Schwaiger. Incarmusic: Context- aware music recommendations in a car. [KJJ18] Mozhgan Karimi, Dietmar Jannach, and In International Conference on Electronic Michael Jugovac. News recommender sys- Commerce and Web Technologies, pages tems - Survey and roads ahead. Informa- 89–100. Springer, 2011. tion Processing and Management, 2018. [BO12] Erik Brynjolfsson and JooHee Oh. The at- [KL17] Ron Kohavi and Roger Longbotham. On- tention economy: Measuring the value of line controlled experiments and a/b test- free digital services on the internet. In ing. In Encyclopedia of Machine Learning ICIS. Association for Information Systems, and Data Mining, pages 922–929. Springer, 2012. 2017. [LA13] Xin Liu and Karl Aberer. Soco: a social map for top-n context-aware recommenda- network aided context-aware recommender tion. In Proceedings of the 35th interna- system. In Proceedings of the 22nd Inter- tional ACM SIGIR conference on Research national Conference on World Wide Web, and development in information retrieval, pages 781–802. ACM, 2013. pages 155–164. ACM, 2012. [Lom14] Andreas Lommatzsch. Real-time news rec- [SKP13] Jeong-woo Son, A-yeong Kim, and Seong- ommendation using context-aware ensem- bae Park. A Location-Based News Article bles. In European Conference on Informa- Recommendation with Explicit Localized tion Retrieval, pages 51–62. Springer, 2014. Semantic Analysis. SIGIR, pages 293–302, [LSC+ 10] Lihong Li, Robert E Schapire, Wei Chu, 2013. John Langford, and John Langford. A contextual-bandit approach to personal- [SSZ18] Gabriele Sottocornola, Panagiotis Syme- ized news article recommendation. In onidis, and Markus Zanker. Session-based the 19th International Conference on the News Recommendations. In the 2018 World Wide Web, pages 661–670, New World Wide Web Conference, pages 1395– York, 2010. ACM Press. 1399, New York, 2018. ACM Press. [LWW15] Qiang Liu, Shu Wu, and Liang Wang. [WRW12] Xinxi Wang, David Rosenblum, and COT: Contextual operating tensor for Ye Wang. Context-aware mobile music context-aware recommender systems. In recommendation for daily activities. In AAAI, pages 203–209, 2015. Proceedings of the 20th ACM international conference on Multimedia, pages 99–108. [LZYL14] Lei Li, Li Zheng, Fan Yang, and Tao Li. ACM, 2012. Modeling and broadening temporal user in- terest in personalized news recommenda- [WZL+ 15] S. Wang, B. Zou, C. Li, K. Zhao, Q. Liu, tion. Expert Systems with Applications, and H. Chen. Crown: A context-aware rec- 41(7):3168–3177, 2014. ommender for web news. In 2015 IEEE 31st International Conference on Data En- [ÖGE14] Özlem Özgöbek, Jon Atle Gulla, and gineering, pages 1420–1423, Seoul, 2015. Riza Cenk Erdur. A survey on challenges IEEE. and methods in news recommendation. In Proceedings of the 10th International Con- [YSC+ 13] Hongzhi Yin, Yizhou Sun, Bin Cui, Zhit- ference on Web Information Systems and ing Hu, and Ling Chen. Lcars: a location- Technologies (WEBIST), volume 2, pages content-aware recommender system. In 278–285, 2014. Proceedings of the 19th ACM SIGKDD in- [SJNP12] Alan Said, Brijnesh J Jain, Sascha Narr, ternational conference on Knowledge dis- and Till Plumbaum. Users and noise: covery and data mining, pages 221–229. The magic barrier of recommender sys- ACM, 2013. tems. In International Conference on User Modeling, Adaptation, and Personaliza- [ZBM12] Yong Zheng, Robin Burke, and Bamshad tion, pages 237–248. Springer, 2012. Mobasher. Differential context relaxation for context-aware travel recommendation. [SKB+ 12] Yue Shi, Alexandros Karatzoglou, Linas In International Conference on Electronic Baltrunas, Martha Larson, Alan Hanjalic, Commerce and Web Technologies, pages and Nuria Oliver. Tfmap: optimizing 88–99. Springer, 2012.