With the change from classical paper-based to online representations of newspapers publishers are able to provide adaptive recommender services supporting users in nding the relevant articles in the huge amount of published news. Since most traditional recommender approaches are tailored to scenarios characterized by static sets of items and exactly identi able users, these approaches cannot be utilized for anonymously consumed news portals o ering a continuously changing set of items. In order to develop an e cient recommender system optimized to the speci c requirements of news portals, we analyze the user behavior and de ne a model for computing recommendations. We have developed tools for continuously monitoring the user preferences and analyzing the features of the most popular news articles. The derived recommender models are implemented in Java using a recommender ensemble architecture that is able to adapt to the speci c characteristics of di erent news portals. The evaluation of the implemented recommender in the CLEF NewsREEL challenge shows that our system provides reliable results and reaches a high Click-Through-Rate. The implemented architecture is open to further optimization with respect to di erent load levels and context parameters.
Current news portals o er huge, continuously changing collections of news articles. In order to support users in nding relevant articles, publishers integrate recommender components suggesting potentially interesting articles to users.
The development of recommender components for news portals leads to several challenges. First, the item sets o ered by news portals change continuously. Second, the identi cation of users does not work precisely since users access the portals anonymously without logging in. Third, the algorithms must be capable of handling high load scenarios and ensure that recommendation requests are successfully answered within short time limits. Moreover, each news portal has its speci c characteristics requiring a customization of the recommender algorithms for the respective domain.
We focus on the news recommendation scenario de ned in the CLEF
NewsREEL challenge [
In Task 2 [
In this work we develop recommender algorithms for news portals addressing the discussed challenges. We analyze the log les from di erent news portals in order to get insights into the user behavior. Based on the identi ed user preferences we de ned a recommender model. The implemented recommender is evaluated both online and o ine in the CLEF NewsREEL challenge. We show that the implemented recommender provides high-quality recommendation results with respect to the CTR and ensures a short response time. Due to the exible architecture of our system, the recommender can be further optimized by dynamically adapting the parameters to the context requirements.
The remaining paper is structured as follows: Section 2 gives a brief overview on existing news recommender algorithms and approaches. Our analysis of the o ine log data and the developed online log analysis tool are explained in Section 3. This section also discusses the derived recommender model and the architecture of the implemented recommender system. The evaluation results are presented in Section 4. Finally, a conclusion and an outlook to future work are given in Section 5. 2
In this Section we review existing approaches to recommender systems and
analyze how these algorithms t with the requirements of our scenario.
User-based Collaborative Filtering Most recommender systems implement
Collaborative Filtering (CF). CF-based algorithms analyze the user feedback (e.g.
ratings user assign to items) [
In the analyzed news recommendation scenario, the sets of users and items change continuously. This leads to a permanent cold start problem. Thus, in our case, user-based CF does not seem to be a suitable approach.
Item- and Content-based Recommender Approaches An alternative to user-based
CF are item-to-item recommender algorithms. These approaches are based on
the idea that users prefer items similar to items they liked in the past. The
similarity between the items can either be computed based on ratings (item-based
CF) or based on the content features of the items (content-based ltering) [
In the news recommendation scenario this approach seems to be promising
due to the fact that it can handle fast changing sets of users and items [
Discussion In the analyzed news recommendation scenario the use of mostpopular algorithms seems to be an adequate approach. In contrast to user-based CF, most-popular algorithms do not su er from the cold start problem and do not require exact user pro les. Di erently from content-based approaches, they analyze the user behavior instead of content features leading to a better recommendation precision. This analysis motivates us to further investigate most-popular algorithms. 3
In this section we rst analyze the o ine log data from the news portals (part of the NewsREEL challenge) to study the trends in the user behavior in detail. Subsequently, we describe the approaches and tools used for de ning a suitable recommender model and discuss the implemented strategy in detail. 3.1
In order to de ne an e cient recommender model we analyze the o ine log data. We assess user behavior with respect to impressions and clicks searching for patterns that indicate the users' preferences.
In a rst step, we analyze the correlation of impressions and clicks. Figure 1 shows that news articles receiving a large number of clicks typically also have a large number of impressions; but a large number of impressions does not necessarily imply that an article receives a high number of clicks. With respect to the development of a recommender algorithm, Figure 1 shows that a large number of impressions is a valuable indicator that an article is interesting and a promising candidate for answering a recommendation request.
In a next step, we analyze the distribution of the number of impressions over the set of articles. Figure 2 shows the number of impression (y-axis) with respect to the rank of impressions (x-axis) for di erent publishers. The impression rank is computed by sorting the article set in descending order of the impression count. The article with the highest number of impressions has an impression rank of zero. The graphs show that there are very few articles with very high 1,E+08 t 1,E+07 un 1,E+06 o cn 1,E+05 o iss 1,E+04 e rp 1,E+03 im1,E+02 1,E+01 1,E+00 0 100 200 300 400 500 600 700 800 900
impression rank impression count while the curve decays very fast with the impression rank. The observation indicates that there are articles of very high interest for most users. Thus recommending the most popular articles seems to be a promising strategy since most users are likely to read these articles. 3.2
The analysis of the o ine log data motivates us to analyze the most popular articles in detail. We develop an online log viewer allowing us the visualization of characteristic features of the most popular articles for each news portal.
The online analysis tool consists of a database and two components, one for aggregating the relevant information from the online message stream and one for visualizing the data. The data aggregation component extracts data relevant for the analysis from the message stream and stores it in a database. The data comprise the article title, the article text, the publisher, and additional meta-data. Furthermore, it aggregates the number of clicks and the number of impressions on a 15 minutes basis.
The visualization component is implemented as a web application based on a Grizzly web server1. The web application uses SQL statements for retrieving the data from the database required in the visualization. The data are converted to the JSON format and sent to the HTML client that places the data on the HTML formatted web page.
The created user interface visualizes the most popular articles for the di erent domains and for a user-de ned timeframe. For each publisher, articles are sorted by the number of impressions within the set time segment. Figure 3 shows a visualization example. For the analysis the time frame and the publisher must be de ned. The tool visualizes the number of impressions for the selected publisher in a histogram. The news articles (most popular in the selected time frame) are shown in a list. For each article the most important information is displayed. 1 https://grizzly.java.net/
Fig. 3. The gure shows the front page of the online monitoring web application.
This includes the title of the article, the articlesID, the number of views within the set time segment and the abstract summarizing the content.
Furthermore, the click on an article opens a new panel showing the impression statistic for each article. The diagram visualizes the number of impressions per hour during the last three days. Figure 4 shows an example chart for a selected article published on May 13th, 2016, at 12 o'clock. The graph shows the high variance in the number of impressions per hour over the day. Another important aspect the chart reveals is that the number of impressions per hour decreases over time. This is due to the reduced interest of users in outdated articles. Based on our analysis, we develop a recommender component implementing a most-popular algorithm. Due to the di erences in the number of articles and average lifetime of the items from di erent portals, we use a meta-recommender architecture having a special component for each domain as shown in Figure 5.
plista
I P A P R O I P A P R O sbaed iIanD iton dom a e leg th de on
domain N online log data aggregator itIeDm coutn
D I m e it map sorted by count list of itemIDs (popularity statistic) (insertion order,
no duplicates) online log database online log viewer
The implemented strategy We implement a most-popular algorithm by analyzing the most recent impressions with regard to their popularity for each domain. For this purpose we use a map (impressionID, number of impressions) storing the number of impressions for each itemID. In order to limit the size of the map, we use a queue (\ fo"), storing the articles' IDs in insertion order. If the maximal number of keys in the map is reached, the oldest key from the map is removed. We implement the map based on a concurrent skip list containing the impressions in descending order of the number of impressions. This data structure has several advantages: (1) The map supports the concurrent access enabling the use of threading. In addition to that, no explicit synchronization is needed. This ensures that an update of the impression statistic does not block recommendation requests. This is important to ensure that we can correspond to the tight time limits for the requests. (2) Since the map contains the itemIDs sorted by popularity, recommendation requests are e ciently answered by returning the head of the list of itemIDs. This minimizes the complexity of providing recommendations.
Discussion In this section we motivated the use of a most-popular model. We presented our online log viewer visualizing the characteristic features as well as the impression statistics for the most popular articles. Moreover, we presented our meta-recommender architecture combining recommender components customized for the di erent news portals (\domains"). Each recommender component uses a most-popular algorithm tailored for the characteristics of the speci c portal.
The implementation of the most-popular recommender components uses highly optimized data structures allowing concurrent access due to the use of concurrent collections. Since the potentially relevant articles are already sorted by popularity, the e cient handling of requests is assured. The meta-recommender architecture ensures that the optimal window size for each portal is used handling the removal of outdated articles. The suitable window size is determined based on the analysis of the log data. 4
We benchmark the implemented news recommender system both in the online and the o ine task.
Online Evaluation Our recommender participated actively in the online evaluation from April 14th until May 20th, 2016. We restrict our evaluation to the domain 35774 due to the observation that 96% of all requests belong to this domain (cf. Fig. 6). The number of requests for the other publishers is small so that the signi cance of the results for the other domains is limited.
Table 1 summarizes the results obtained in the online task. The results show that the implemented strategy works very reliably. Our recommender handled a large number of requests and, having a median response rate of 99.4%, signi cantly outperformed the baseline recommender.
In order to analyze the online task in detail, we plot the CTR on a daily basis. Figure 7 shows the number of requests and the reached CTR. The results show that the CTR is stable. On May 3rd, 2016, the CTR dropped signi cantly. Since the recommender has not been changed during the entire evaluation, the decrease must be caused by external factors that cannot be in uenced by the recommender algorithms. In order to handle external changes, the recommender should be able to adapt itself to new environments.
O ine Task In addition to the online evaluation, we also benchmark the performance of the recommender system o ine based on a simulated stream. We replay the data stream recorded in one week (May 9th - May 15th, 2016) and analyze the o ine CTR as well as the response time.
Figure 8 visualizes the o ine CTR of our algorithm. Compared to the online CTR the o ine CTR is slightly lower. This indicates that our algorithm is optimized for the online reward model. The user impressions (used as reward model in the o ine evaluation) seem to follow di erent patterns than the user clicks (used as reward model in the online evaluation). The di erences between the online and the o ine evaluation results could be explained by the observation that user impressions cover a wide spectrum of articles but users only click on recommendations if special (very popular) articles are recommended.
We also evaluated the response time of the recommender algorithm. The evaluation has been executed on a laptop running Windows 7 having an Intel R CoreTM i7-3520M 2.9GHz CPU and 16 GB of RAM. Figure 9 shows the response time for di erent levels of concurrent requests. The results show that the implementation e ciently handles requests. 1.6% 1.4% 1.2% 1.0% 0.8% 0.6% 0.4% 0.2% 0.0%
Requests Clicks
CTR # correct recommendations *100 # provided recommendations offline CTR 3% 2% 1% 0%
R T C e n lif f o In this paper we presented our news recommender system tailored to the speci c requirements of the NewsREEL challenge. Our system uses a most-popular recommender approach that adapts to the characteristics of the di erent news portals (\publishers"). In order to motivate that a most-popular algorithm is an e cient solution for the news recommendation task, we developed an online tool visualizing the most popular articles and their characteristics. Our analysis showed that the most popular articles cover a high fraction of the user-item interactions. The user behavior di ers between the di erent portals. This has been the motivation for developing a recommender implementing a meta-recommender architecture combining components that are optimized for the relevant news portals.
In order to handle the problem of continuous changes in the user interests, we implemented a sliding window approach ensuring that only the most recent popular articles are recommended. Outdated articles are automatically removed from the set of candidates considered when computing recommendations. The evaluation showed that the implemented recommender outperforms the baseline for all considered domains. The error rate is very low. The response time requirements are reliably ful lled.
The analysis of the user behavior showed that the optimal window size considered while computing the most popular news articles depends on the news portal. As future work we plan to apply a context-aware optimization of the window size in order to take into account the day of the week (working day, weekend, and holiday) and the time of the day. Furthermore, we plan to implement algorithms predicting what articles will be popular in the next hour. For this purpose we identify current trends and extrapolate these trends into the near future. The research leading to these results was performed in the CrowdRec project, which has received funding from the European Union Seventh Framework Programme FP7/2007-2013 under grant agreement No. 610594.