Parsimonious and Adaptive Contextual Information Acquisition in Recommender Systems Matthias Braunhofer Ignacio Francesco Ricci Faculty of Computer Science Fernández-Tobías Faculty of Computer Science Free University of Escuela Politécnica Superior Free University of Bozen-Bolzano Universidad Autónoma de Bozen-Bolzano Bozen-Bolzano, Italy Madrid Bozen-Bolzano, Italy mbraunhofer@unibz.it Madrid, Spain fricci@unibz.it ignacio.fernandezt@uam.es ABSTRACT challenges [2]. First, it is necessary to identify the con- Context-Aware Recommender System (CARS) models are textual factors that could potentially influence individual’s trained on datasets of context-dependent user preferences preferences (ratings) and the decision-making process, and (ratings and context information). Since the number of hence are worth to be collected, either automatically (e.g., context-dependent preferences increases exponentially with the time, or the location), or by querying the user. The the number of contextual factors, and certain contextual in- second challenge is to develop a predictive model that is formation is still hard to acquire automatically (e.g., the capable of predicting the users’ ratings for items under var- user’s mood or for whom the user is buying the searched ious contextual situations. Finally, the design of a proper item) it is fundamental to identify and acquire those factors human-computer interaction layer on top of the predictive that truly influence the user preferences and the ratings. In model is the third and last but not least challenge for build- particular, this ensures that (i) the user effort in specifying ing a CARS. contextual information is kept to a minimum, and (ii) the In this paper we are focusing on the first challenge. In system’s performance is not negatively impacted by irrele- this respect, previous approaches have mainly applied fea- vant contextual information. In this paper, we propose a ture selection techniques to identify which contextual factors novel method which, unlike existing ones, directly estimates should be used in the rating prediction phase. The downside the impact of context on rating predictions and adaptively of this approach is that it may force users to add to ratings identifies the contextual factors that are deemed to be useful contextual information that later on, when the prediction to be elicited from the users. Our experimental evaluation model is built, may be found not to be useful for improv- shows that it compares favourably to various state-of-the-art ing the system performance. Because of that, here we pro- context selection methods. pose a new method for identifying which contextual factors should be acquired from the user upon rating an item, so that the user will not enter the value of many contextual Categories and Subject Descriptors factors (parsimonious), and the accuracy of the subsequent H.3.3 [Information Storage and Retrieval]: Information recommendations is improved the most. Search and Retrieval—information filtering As a concrete motivation, consider the places of interest (POIs) CARS that is illustrated in Figure 1 and Figure 2 Keywords [5]. That system is called STS (South Tyrol Suggests) and it uses 14 contextual factors (e.g., weather, mood, distance, Context-Aware Recommender Systems; Contextual Infor- time available). Users may specify any of them when en- mation; Feature Selection tering a rating for a POI (and also when the user requests context-aware recommendations). These are, however, not 1. INTRODUCTION all equally important for different user-item pairs, in the Context-Aware Recommender Systems (CARSs) gener- sense that they contribute differently to the improvement ate more relevant recommendations than traditional Rec- of the system’s rating prediction and recommendation ac- ommender Systems (RSs) by adapting them to the specific curacy. In fact, we must avoid any possible waste of time contextual situation of the user (e.g., time, weather, loca- and effort of the user while entering this information and tion) [1]. The development of an effective CARS faces many also keep away from the potential degradation of the system performance that could be caused by the usage of irrele- vant information. For example, the user’s mood may be extremely important to predict the ratings only of certain users, and weather may be an essential factor for one class of items, while negligible for others. Unlike current state-of-the-art strategies, which measure the relevance of contextual factors on a global basis, our strategy dynamically and adaptively selects the contextual factors to be elicited from the user when she enters a rat- . ing for an item. This is achieved by using the CARS rating be considered by the system, or b) for selecting, a posteri- prediction model itself, and asking the user to specify, when ori, after the ratings and context data was acquired, those she is rating an item, those contextual factors that if consid- factors that are most influential for computing rating pre- ered in the model would produce a rating prediction for that dictions. The first task was tackled by exploiting domain item that is most different from the prediction computed by knowledge of the RS’s designer or market expert [2], whereas a context-free model. We consider this as a heuristics: if this the second one was addressed by using feature selection al- contextual information has an impact on rating prediction gorithms. it should be acquired and used in the model. In order to tackle the second task, Odić et al. [13] provide Several CARS algorithms can be used to implement the several statistical measures for relevant-context detection above mentioned solution; here we employ a new variant of (i.e., unalikeability, entropy, variance, χ2 test and Freeman- Context-Aware Matrix Factorization (CAMF) [3] that lever- Halton test), and show that there exists a significant differ- ages latent correlations and patterns between users, items ence in the prediction of ratings when using relevant and as well as contextual conditions, thus making it well-suited irrelevant context. Another example can be found in [16], for selective context acquisition, but also for prediction and where a Las Vegas Filter (LVF) algorithm [12] is employed: recommendation as well. We have compared our proposed it repeatedly generates random subsets of contextual factors, method with several state-of-the-art context selection strate- evaluates them based on an inconsistency criterion and fi- gies in an offline experiment on two contextually-tagged rat- nally returns the subset with the best evaluation measure. ing datasets. The results show that the proposed parsimo- Finally, Zheng et al. [17] presented a set of approaches based nious and personalized acquisition of relevant contextual fac- on multi-label classification for the task of recommending tors is efficient and effective, and allows to elicit ratings aug- the most suitable contexts in which a user should consume mented with contextual factor values that best improve the a specific item. recommendation performance in terms of accuracy, precision Rather than post filtering (after the rating data was ac- and recall. quired) the contextual factors in the rating prediction phase, We note that parsimoniously acquiring from the user rele- we are interested in detecting which contextual factors should vant contextual information can be considered as an Active be acquired upfront from the user in the first place. Hence, Learning problem [8]. But, while in previous work [6, 4] we when a specific user rates a particular item, our goal is to focused on the active identification of the items to present to parsimoniously request and possibly elicit only the contex- the user to rate, in this article we focus on the subsequent de- tual factors that improve the most the system performance. cision of identifying which contextual factors the user should These factors can differ for each user-item pair. Moreover, enter, i.e., under which conditions the user experienced the instead of relying on statistical measures, which has been item. the major trend so far, our work uses a CARS rating pre- The rest of the paper is structured as follows. In Sec- diction model itself to estimate the usefulness of contextual tion 2, we review the related work. Section 3 introduces factors. Our approach is similar to some Active Learning our main application scenario. Section 4 presents in detail [8] solutions of the cold-start problem that also use the rat- the proposed context acquisition method. Then, we describe ing prediction model to identify which items are better to the experimental evaluation in Section 5, and detail the ob- propose to the users to rate. An example of such an Active tained results in Section 6. Finally, conclusions are drawn Learning method can be found in [10]; it asks users to rate and future work directions are described in Section 7. the items whose ratings, if known, contribute most to reduce the system prediction error on a set of held-out test ratings. 2. RELATED WORK Another similar approach is the influence-based method pre- sented in [15], which selects those items whose ratings are Finding the most relevant features for building a predic- estimated to have the highest influence on the rating predic- tion model has been extensively studied in machine learning. tions of other items. Feature selection is aimed at improving the performance of learning algorithms and gaining insight into the unknown generative process of the data [9]. There are three main 3. APPLICATION SCENARIO approaches to feature selection: wrappers, filters and em- Our application scenario is a mobile CARS called STS bedded methods. While wrapper methods optimize the se- (South Tyrol Suggests) [5] that is available on Google Play lection within the prediction model, filter methods employ Store and recommends POIs to visit in the South Tyrol re- statistical characteristics of the training data to select fea- gion of Italy. STS can generate POI recommendations (Fig- tures independently of any prediction model, and thus are ure 1, left) adapted to the user’s and items’ current con- substantially faster to compute. Popular examples of filter textual situation by exploiting 14 contextual factors whose methods used in machine learning include mutual informa- conditions (values) are partially acquired automatically by tion, t statistic in Student test, χ2 test for independence, F the system (e.g., weather at the POI, season, daytime) and statistic in ANOVA and minimum Redundancy Maximum partially entered manually by the user through an appro- Relevance (mRMR) [14], which uses the mutual information priate screen (e.g., user’s budget, companion, feeling), as of a feature and a class as well as the mutual information shown in Figure 2 (right). More information about the used of features to infer features’ relevance and redundancy, re- contextual factors and their possible values, which are called spectively. Differently from the two previous methods, em- contextual conditions, can be obtained from Table 1. The bedded methods use internal parameters of some prediction user’s preference model is learned using a set of in-context model to perform feature selection (e.g., the weight vector ratings that the system actively collects from the users and in support vector machines). that describe the users’ evaluations for the POIs together Focussing now on CARSs, previous research has explored with the contextual situations in which the users visited the methods: a) for identifying a priori the factors that should POIs (see Figure 2). However, in our application scenario, given the relatively large number of contextual factors we faced the problem of choosing the contextual factors to ask to the end user upon rating a POI. This is an important and practical problem: asking the value of all the contex- tual factors is not effective, as it would take too much time and effort for the user to specify them. Moreover, asking the wrong subset of contextual factors may result in the degra- dation of the prediction model performance and in poor rec- ommendations. In order to cope with this problem we propose here a novel method that is able to dynamically and adaptively identify the most important contextual factors to be elicited from a specific user upon rating a particular POI. This method serves the purpose of minimizing the amount of information that the users have to input manually, while at the same time allowing the system to still obtain all the relevant infor- mation needed to maintain a high level of rating prediction performance. Referring to Figure 2, by means of our pro- posed method, we can identify for instance the three most relevant contextual factors for ”Restaurant Pizzeria Amadè” and then present the user with three screens that step-by- step elicit the contextual conditions for these factors. Oth- Figure 2: Rating interface of STS erwise, the user would be required to go through 14 screens, one for each available contextual factor. Table 1: Contextual factors used in STS Contextual Associated contextual conditions factors Weather Clear sky, sunny, cloudy, rainy, thunder- storm, snowing Season Spring, summer, autumn, winter Budget Budget traveler, high spender, none of them Daytime Morning, noon, afternoon, evening, night Companion Alone, with friends/colleagues, with fam- ily, with girlfriend/boyfriend, with chil- dren Feeling Happy, sad, excited, bored, relaxed, tired, hungry, in love, loved, free Weekday Working day, weekend Travel goal Visiting friends, business, religion, health care, social event, education, scenic/landscape, hedonistic/fun, activ- ity/sport Transport No transportation means, a bicycle, a mo- torcycle, a car, public transport Knowledge New to area, returning visitor, citizen of of the travel the area area Crowdedness Crowded, some people, almost empty Figure 1: Context-aware suggestions in STS Duration of Some hours, one day, more than one day stay Temperature Burning, hot, warm, cool, cold, freezing Distance Far away (over 3 km), nearby (within 3 4. SELECTIVE CONTEXT ACQUISITION km) Before presenting the proposed selective context acquisi- tion method, we introduce the CARS predictive model that we have adopted in this study. It is a new variant of the ture latent correlations and patterns between a potentially context-aware predictive model CAMF [3] that treats con- wide range of knowledge sources (e.g., users, items, contex- textual conditions similarly to either item or user attributes tual conditions, demographics, item categories), making it and uses a distinct latent factor vector corresponding to each ideal to derive the usefulness of contextual factors in rating user- and item-associated attribute. More specifically, a con- prediction. Given a user u with user attributes A(u), an textual condition is treated as a user attribute if it corre- item i with item attributes A(i) and a contextual situation sponds to a dynamic characteristic of a user, e.g., the mood, consisting of the conjunction of individual contextual con- budget or companion of the user, whereas it is considered ditions c1 , ..., ck that can be decomposed into the subset of as an item attribute if it describes a dynamic characteris- user-related contextual conditions C(u) and the subset of tic of the item, e.g., the weather and temperature at the item-related contextual conditions C(i), it predicts a rating POI. The model is scalable and flexible, and is able to cap- using the following rule: r̂Alice Skiing = 3.5). Furthermore, assume that 20% of the X X ratings in the rating dataset are tagged with Sunny weather. r̂uic1 ,...,ck = (qi + xa )> (pu + yb )+r̄i +bu Then, the impact of Sunny weather on the user-item pair a∈A(i)∪C(i) b∈A(u)∪C(u) (Alice, Skiing), i.e., ŵAlice Skiing Sunny , is 0.3 (0.2 · |5 − 3.5|). (1) Finally, these individual scores for the contextual condi- where qi is the latent factor vector associated to item i, tions are aggregated into a single relevance score for the con- pu is the latent factor vector associated to user u, xa is textual factor Cj by simply computing the arithmetic mean the latent factor vector associated to an attribute of the of the scores of the various conditions/values for that contex- item i, that may either describe a conventional attribute tual factor. We conjectured that the contextual factors with (e.g., genre, item category) or a contextual attribute (e.g., largest estimated deviation are more useful to optimize the weather, temperature), yb is the latent factor vector asso- system performance. Note that this is quite similar to the ciated to an (contextual or not) attribute of the user u. influence-based Active Learning strategy proposed in [15], Finally, r̄i is the average rating for item i, and bu is the which estimates the influence of an item’s rating on the rat- bias associated to the user u, which indicates the observed ing predictions of other items, and selects the items with the deviation of user u’s ratings from the global average. largest influence for rating acquisition. CARSs can generate recommendations only after having gathered ratings from the users that are augmented with 5. EXPERIMENTAL EVALUATION information about the contextual conditions (values of the contextual factors) observed at the time the item was expe- 5.1 Datasets rienced and rated. It is, however, not always easy to identify In order to evaluate the proposed selective context acqui- which contextual information should be requested and ac- sition method, we have considered two contextually-tagged quired from the users upon rating an item, given the numer- rating datasets with different characteristics. Table 2 pro- ous conditions that might or might not be relevant to pre- vides some descriptive statistics of both datasets. dict new ratings (in various contextual situations). This is where parsimonious and adaptive context acquisition comes • The CoMoDa movie-rating dataset was collected by in. Parsimonious and adaptive context acquisition aims at Odić et al. [13]. It consists of ratings acquired in predicting, for a given user-item pair, the most useful con- contextual situations that are described by the con- textual factors, i.e., those that when elicited together with junction of multiple conditions coming from 12 differ- the rating from the user improve more the quality of future ent factors, for instance, time, daytype, season and recommendations, both for that user and for other users of mood. In addition to the ratings data, this dataset the system. also includes well-defined user attributes (i.e., age, gen- As we mentioned in the related work section, there exist der, city, country) and movie attributes (i.e., director, many algorithms that even though principally designed for country, language, year, budget, genres, actors). context / feature selection (i.e., selection of the most useful contextual factors / features to be used for prediction) can • The TripAdvisor dataset is a dataset that we crawled be used also for the purpose of parsimonious context acqui- from the TripAdvisor1 website, which is one of the sition (i.e., selection of the contextual factors to be elicited largest travel sites in the world. It contains ratings for from the user upon rating an item). In this paper, we pro- POIs in the South Tyrol region of Italy that are tagged pose a new strategy, which we call Largest Deviation. Differ- with contextual situations described by the conjunc- ently from several state-of-the-art context / feature selection tion of contextual conditions coming from three con- strategies, it personalizes the selection of the contextual fac- textual factors, namely, type (e.g., couple, family or tors to ask to the user when rating an item by computing a business trip), month (e.g., January, February) and personalized relevance score for a contextual factor Cj and year (e.g., 2015, 2014) of the trip. Additionally, also user-item pair (u, i). To achieve this, for each user u and the TripAdvisor dataset has well-defined user (e.g., item i pair (whose rating is acquired) we first measure the user location, member type) and POI attributes (e.g., “impact” of each contextual condition cj ∈ Cj , denoted as item type, amenities, item locality). ŵuicj , by calculating the absolute deviation between the rat- We note that other rating datasets, which are commonly ing prediction when the condition holds (i.e., r̂uicj ) and the used in CARS research, are not suitable for our analysis since predicted context-free rating (i.e., r̂ui ): they contain ratings augmented only with the knowledge of a subset of all the contextual factors. For instance, in ŵuicj = fcj |r̂uicj − r̂ui |, (2) STS, the POIs RS that we mentioned in Section 3, when a user rates a POI she commonly specifies only the value where fcj denotes the normalized frequency of the contex- of two or three of the fourteen contextual factors that the tual condition cj , and is calculated as the fraction of ratings system manages (see Table 1). The lack of knowledge of in the entire dataset that are tagged with contextual condi- all the contextual factors for each rating is a problem in |Rc | tion cj (i.e., |R|j ). The normalized frequency adjusts the our case, because, as we will describe in Section 5.2, we raw absolute deviation by taking into account that the con- wanted to simulate a rating acquisition process where, for textual conditions with largest frequency are more reliable. a given item, the system requests the user to rate it and to For example, suppose that you want to estimate the impact enter the values of the contextual factors identified by the of Sunny weather on the user-item pair (Alice, Skiing). Let proposed method. Therefore, every contextual factor must us assume that the rating prediction for Alice of Skiing is 5 be available in the dataset in order to be acquired during under Sunny weather (i.e., r̂Alice Skiing Sunny = 5), and that the simulated interactions. 1 the corresponding context-free rating prediction is 3.5 (i.e., http://www.tripadvisor.com/ contextual factor Cj as the normalized mutual infor- Table 2: Datasets’ characteristics mation between the ratings for items belonging to i’s Dataset CoMoDa TripAdvisor Domain Movies POIs category and Cj ; the higher the mutual information, Rating scale 1-5 1-5 the better the contextual factor can explain the user Ratings 2,098 4,147 ratings for items of a particular category. We note that Users 112 3,916 this strategy depends on the item category but is not Items 1,189 569 personalized, i.e., the same contextual factors are re- Contextual factors 12 3 Contextual conditions 49 31 quested to any user upon rating an item belonging to User attributes 4 2 a particular category. Item features 7 12 • Freeman-Halton Test: proposed as context selection strategy in [13], it calculates the relevance of a con- textual factor Cj using the Freeman-Halton test. The 5.2 Evaluation Procedure Freeman-Halton test is the Fisher’s exact test extended to contingency tables larger than 2×2, which is a com- In the evaluation we have simulated system/user inter- mon alternative to the χ2 test in case the Cochran’s actions where the users rate items specifying only the val- rule about small expected frequencies is not satisfied. ues of contextual factors (contextual conditions) that have The null hypothesis of the test is that the contextual been identified by a context selection strategy. To achieve factor Cj and the ratings are independent. If the null this, we adapted a procedure which was employed to eval- hypothesis can be rejected, one can conclude that the uate Active Learning strategies for RSs [7]. This procedure contextual factor Cj and the ratings are dependent and first randomly partitions all the available ratings into three thus that the contextual factor Cj is relevant. This test subsets in the ratio 25:50:25%, respectively: (i) training set is performed on the full dataset and therefore the se- that contains the ratings that are used to train the con- lected factors do not depend on the user or the item text acquisition strategies; (ii) candidate set containing the to be rated. ratings that can be potentially transferred into the train- ing set with the contextual conditions matched by the con- • Minimum Redundancy Maximum Relevance (mRMR): text acquisition strategies; and finally (iii) testing set which mRMR [14] is a widely used feature selection algo- contains the part of the ratings that is withheld from the rithm, which, to the best of our knowledge, has not yet system in order to calculate various performance metrics, been used for the purpose of context selection. It ranks i.e., user-averaged MAE (U-MAE), Precision@10 and Re- each contextual factor Cj according to its relevance to call@10. Then, for each user-item pair (u, i) in the candi- the rating variable and redundancy to other contex- date set, the N most relevant contextual factors according tual factors, where both relevance and redundancy are to a context usefulness strategy are computed, with N (in measured based on mutual information. Analogous to different experiments) varying from 1 to the total number the Freeman-Halton test, it is calculated on the full of contextual factors in the rating dataset, and the corre- dataset and the selected factors are used for all user- sponding rating ruic in the candidate set is transferred to item rating combinations. the training set as ruic0 with c0 ⊆ c containing the associ- • Random: the score for a contextual factor Cj is simply ated contextual conditions for these contextual factors. For a random float in the interval [0, 1). Hence, the top instance, if the top two contextual factors for the user-item N contextual factors for a user-item pair are simply pair (Alice, Skiing) are Season and W eather, and Alice’s randomly chosen. This is a baseline strategy used for rating is rAlice Skiing W inter,Sunny,W arm,M orning = 5, then comparison. rAlice Skiing W inter,Sunny = 5 is added to the training set. Since in the considered rating datasets all the contextual factors were specified for each rating, we could always ac- quire the contextual conditions for the top contextual fac- Table 3: Overview of tested strategies for selective tors. Finally, the evaluation metrics were measured on the context acquisition testing set, after training the rating prediction model on the new extended training set. Strategy User Item The above process was repeated 20 times with different Personalization Dependence random seeds and the results were averaged over the splits Largest Deviation 3 3 to yield more robust estimates (i.e., repeated random sub- Mutual Information 7 3 sampling validation [11]). Freeman-Halton Test 7 7 mRMR 7 7 5.3 Baseline Methods for Evaluation Random 7 7 We have compared the performance of our proposed Largest Deviation method with the following three state-of-the-art context / feature selection strategies, in addition to Random which we used as a baseline (see Table 3 for a summary of 6. EVALUATION RESULTS all the tested methods): Figure 3 and Figure 4 show the U-MAE, Precision@10 and Recall@10 results of the CARS algorithm obtained by • Mutual Information: the usage of mutual information applying the various context acquisition strategies on the for context selection was proposed in [2]. Given a user- CoMoDa and TripAdvisor dataset, respectively. In the fig- item pair (u, i), it computes the relevance score for ures, the x-axis represents the number of acquired contextual Figure 3: Accuracy, precision and recall results for Figure 4: Accuracy, precision and recall results for the CoMoDa dataset the TripAdvisor dataset factors, and statistically significant improvements (paired t- that in this strategy, every contextual factor has the same test, p < 0.05) of the proposed Largest Deviation strategy chance of being selected. As a side effect, this allows to bet- over the other considered strategies are indicated by aster- ter explore the effect of individual contextual conditions on isks on top of the bars. On the CoMoDa dataset, by us- users and/or items. However, the Random strategy cannot ing up to three contextual factors, Largest Deviation strat- be practically used since it can often request meaningless egy can achieve a significantly better performance in terms contextual factors to the user, e.g., the budget for a POI of U-MAE, Precision@10 and Recall@10 when compared that can be visited for free. Hence, the random strategy is with the other strategies, i.e., Mutual Information, Freeman- not directly applicable in a realistic scenario and can only Halton Test and mRMR. With four contextual factors se- be used in combination with other strategies. This is in lected, however, there is a notable increase in the U-MAE line with the findings of Elahi et al. [7], who suggested to of Largest Deviation, which also causes Precision@10 and consider “partially randomized” strategies that add a small Recall@10 to drop. We note that in the graph the num- portion of randomly selected items to those identified by ber of selected contextual factors goes only up to 4 (out of another baseline strategy. 12) in order to focus the presentation on the selection of a Looking at the results for the TripAdvisor dataset, one small subset of factors. In fact, the performance differences can note that minor differences (especially in Precision@10 between the strategies vanish when more than 4 contextual and Recall@10) between the considered context acquisition factors are acquired. We also note that all these 12 con- strategies are present. This is due to the fact that in this textual factors were supposed to be relevant in the movie dataset in total only three contextual factors are available, recommendation domain [13]. Hence our results clearly in- thus providing only little potential for parsimonious and dicate that a parsimonious context acquisition strategy is adaptive contextual factor selection. Nevertheless, it can highly beneficial. be seen that Largest Deviation achieves even here a very Experimental results also indicate that the Random strat- good accuracy for the tested number of selected contextual egy has a relatively good performance. Our explanation is factors (1 - 3). 7. CONCLUSIONS AND FUTURE WORK recommender systems. In Learning and Collaboration In this paper, we have proposed a new method for parsi- Technologies. Technology-Rich Environments for monious context acquisition, i.e., for identifying, for a given Learning and Collaboration, pages 105–116. Springer, user-item pair the contextual factors that when acquired to- 2014. gether with the rating from the user let the system to gener- [5] M. Braunhofer, M. Elahi, and F. Ricci. Usability ate better predictions. This is an important and challenging assessment of a context-aware and personality-based problem for CARSs, since usually many contextual factors mobile recommender system. In E-Commerce and (e.g., location, weather, time of day, mood) may be available, Web Technologies, pages 77–88. Springer, 2014. but only a small subset may be useful and should be asked [6] M. Elahi, M. Braunhofer, F. Ricci, and M. Tkalcic. to the user to avoid an unnecessary waste of time and effort Personality-based active learning for collaborative as well as to avoid any degradation of the recommendation filtering recommender systems. In AI* IA 2013: model performance. Advances in Artificial Intelligence, pages 360–371. We have formulated the experimental hypothesis that the Springer, 2013. proposed parsimonious and personalized selective context [7] M. Elahi, F. Ricci, and N. Rubens. Active learning acquisition strategy is able to elicit ratings with contex- strategies for rating elicitation in collaborative tual information that improve more the recommendation filtering: a system-wide perspective. ACM performance in terms of accuracy, precision and recall, and Transactions on Intelligent Systems and Technology also compares favourably with state-of-the-art (context se- (TIST), 5(1):13, 2013. lection) alternatives. In an offline experiment on two rating [8] M. Elahi, F. Ricci, and N. Rubens. Active learning in datasets we were able to confirm these hypotheses. collaborative filtering recommender systems. In Selective context acquisition is still a new and under- E-Commerce and Web Technologies, pages 113–124. researched topic, and there are some research questions that Springer, 2014. deserve future work. Firstly, what is the effect on system [9] I. Guyon and A. Elisseeff. An introduction to variable performance of employing an Active Learning method for and feature selection. Journal of Machine Learning adaptively selecting both the item to rate and the contex- Research, 3:1157–1182, 2003. tual information to add. In this paper we have addressed [10] R. Karimi, A. Nanopoulos, and L. Schmidt-Thieme. A only partially the problem, by identifying the contextual fac- supervised active learning framework for recommender tors that should be acquired, when a user is rating an item. systems based on decision trees. User Modeling and Secondly, it is interesting to understand how the proposed User-Adapted Interaction, 25(1):39–64, 2015. selective context acquisition method can be extended to gen- [11] R. Kohavi et al. A study of cross-validation and erate requests for contextual data that takes into account the bootstrap for accuracy estimation and model selection. possible correlation between contextual factors. Thirdly, it In Ijcai, volume 14, pages 1137–1145, 1995. would be interesting to update the evaluation procedure so [12] H. Liu, R. Setiono, et al. A probabilistic approach to that it can be used also on datasets of contextually-tagged feature selection-a filter solution. In ICML, volume 96, ratings for which only a subset of the contextual factors is pages 319–327. Citeseer, 1996. known; as it occurs in the rating dataset collected by our STS app. Finally, we plan to integrate the developed con- [13] A. Odić, M. Tkalčič, J. F. Tasič, and A. Košir. text acquisition method into our STS app so that we can Predicting and detecting the relevant contextual perform a live user study and assess the impact and the information in a movie-recommender system. benefit of the proposed dynamic and personalized parsimo- Interacting with Computers, 25(1):74–90, 2013. nious acquisition of contextual factors. [14] H. Peng, F. Long, and C. Ding. Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. 8. REFERENCES Pattern Analysis and Machine Intelligence, IEEE [1] G. Adomavicius, B. Mobasher, F. Ricci, and A. Tuzhilin. Context-aware recommender systems. AI Transactions on, 27(8):1226–1238, 2005. Magazine, 32(3):67–80, 2011. [15] N. Rubens and M. Sugiyama. Influence-based [2] L. Baltrunas, B. Ludwig, S. Peer, and F. Ricci. collaborative active learning. In Proceedings of the Context relevance assessment and exploitation in 2007 ACM Conference on Recommender Systems, mobile recommender systems. Personal and pages 145–148. ACM, 2007. Ubiquitous Computing, 16(5):507–526, 2012. [16] B. Vargas-Govea, G. González-Serna, and [3] L. Baltrunas, B. Ludwig, and F. Ricci. Matrix R. Ponce-Medellın. Effects of relevant contextual factorization techniques for context aware features in the performance of a restaurant recommendation. In Proceedings of the Fifth ACM recommender system. ACM RecSys, 11, 2011. Conference on Recommender Systems, pages 301–304. [17] Y. Zheng, B. Mobasher, and R. Burke. Context ACM, 2011. recommendation using multi-label classification. In [4] M. Braunhofer, M. Elahi, M. Ge, and F. Ricci. Proceedings of the 13th IEEE/WIC/ACM Context dependent preference acquisition with International Conference on Web Intelligence, pages personality-based active learning in mobile 288–295. IEEE/WIC/ACM, 2014.