Since the users' decision making depends on the situation the user is in, contextual information has shown to improve the recommendation procedure in context-aware recommender systems (RS). In our previous work we have shown that relevant contextual factors have signi cantly improved the quality of rating prediction in RS, while the irrelevant ones have degraded the prediction. In this work we focus on the detection of relevant contextual conditions (i.e., values of contextual factors) which in uence the users' decision making process. The goals are (i) to lower the intrusion for the end user by simplifying the acquisition process, and (ii) to reduce the sparsity of the acquired data during the contextual modeling. The results showed signi cant improvement in the rating prediction task, when managing the irrelevant contextual conditions by the approach that we propose in this paper.
Over the past decade, employing contextual information
in recommender systems (RS) has been a popular research
topic. Contextual information is de ned as the information
that can be used to describe the situation and the
environment of the entities involved in such systems [
In this work we follow the terminology described in [
In our previous work [
In this work we focus on the detection of relevant contextual conditions, i.e., the values of contextual factors, which in uence the users' decision making process, with the goal of lowering the intrusion for the end user by simplifying the acquisition process, and to reduce the sparsity of the acquired data during the contextual modeling. 1.1
One of the main problems with contextual factors with many contextual conditions is the sparsity of rating data. For example, let us say a speci c user rated 20 items in different contextual situations. For uncontextualized modeling that would be a fair amount of ratings from that speci c user. However, let us say some contextual factor contains ten contextual conditions and users ratings are equally distributed across those conditions. That would mean that for each condition we only have two ratings from that user. For this reason it would be better to have a lower number of contextual conditions per contextual factor.
In addition, since the contextual data is often explicitly
acquired through questionnaires (e.g. in [
To summarize, the acquisition and usage of contextual factors with many contextual conditions has two negative sides: questionnaire size (e ort required from a user) sparsity (ratings are distributed in many categories) Therefore, it would be bene cial to reduce the number of contextual conditions of the relevant contextual factors. 1.2
The problem with the reduction of the number of contextual condition is how to select the conditions to remove and how to merge the contextual conditions in order to reduce their number.
By avoiding the relevant contextual conditions we might lose valuable information. Hence, we need to detect irrelevant conditions, identify how they should be merged and handled during the acquisition, and during the training and the preparation of recommendations.
In this article we propose an approach by which we achieve the following goals: we identify contextual conditions which should be avoided or merged in questionnaires we manage irrelevant categories during training to utilize provided ratings and decrease the sparsity
In the following sections we describe the approach, dataset used and the experimental results. 1.3
In this subsection we describe the experimental design used in this study. For each contextual variable available in the dataset we do the following steps in order to manage irrelevant categories.
First we do the contextual-condition-relevancy detection. At this stage we use statistical testing in order to detect which contextual conditions of a speci c contextual factor are irrelevant and do not have the impact on the ratings. We consider a contextual condition to be relevant if the users' behavior (how users rate items) is di erent for that condition than for other conditions. If the users do not rate items di erently for that contextual condition than otherwise, we consider the condition to be irrelevant.
The next step is to determine whether these irrelevant conditions could be merged with the relevant ones. For example, if rainy weather would be detected as irrelevant, but cloudy weather as relevant, perhaps they could be merged into a combined category cloudy/rainy weather. Hence, we call this step the context-categories-merging determination. Once the merging possibilities are determined, we may use them for two separate tasks: (i) improving the questionnaire, and (ii) improving the contextualized model of users decisions.
Improving the questionnaire. If in a system, after a su cient amount of data was collected, it is determined that several contextual-factors' conditions are irrelevant and could be merged with others, the questionnaire used for the data acquisition should be modi ed. (Similarly, if the data is being collected implicitly through sensors, the acquisition procedure should be modi ed). In this way the number of questions in the questionnaire could be reduced and thus the time required from users to ll-in these questionnaires would be reduced. However, it might be the case that the merges are too complex to employ them in the questionnaire as we will show in the following sections.
Improving the model. In addition to improving the questionnaire, merges should be employed in the model as well. By using the irrelevant conditions in the model during training, the rating data is being used to train the contextualized parameters which depend on the irrelevant contextual conditions. Instead these ratings should be used for training the parameters that depend on the relevant contextual conditions. Hence, by merging categories we are able to use the rating data for the more meaningful task (which consequently reduces the sparsity of ratings), which would result in a better trained model. We will evaluate this task by comparing the root mean square error (RMSE) of the rating prediction with and without merging of context categories, and the results form random merging of contextual conditions as a baseline.
Figure 1 shows the whole procedure described in the article.
Contextual factor C1
C2
C3
C4
C5 …
Cn … … … …
Cn Yes Cn Cn C1 Yes
C2 No
C3 Yes
C4 Yes
C5
No contextual-conditionsrelevancy detection
Contextual factor contextual-conditionsmerges determination
Contextual factor C1 C1
C2 C2 + C1
C3 C3
C4 C4
C5
C5 + C3 improving the questionnaire improving the model
In this section we describe the dataset used in this study and describe each step of the experimental design in more details. 2.1
For the purposes of this work we have used the Context
Movie Dataset (LDOS-CoMoDa), that we have acquired in
our previous work [
We have created an online application for rating movies which users are using in order to track the movies they watched and obtain the recommendations (www.ldos.si/ recommender.html). Users are instructed to log into the system after watching a movie, enter a rating for a movie and ll in a simple questionnaire created to explicitly acquire the contextual information describing the situation during the consumption.
The part of the dataset used in this study consists of 1611
ratings from 89 users to 946 items with 12 associated
contextual factors. Additional information about our Context
Movies Database (LDOS-CoMoDa) can be found in [
All the contextual factors and conditions acquired are listed in Table 1. In order to determine if a contextual condition of a speci c contextual factor is relevant, we use the Wilcoxon ranksum test in the following way. For each condition (e.g., sunny weather) of a speci c contextual factor (e.g., weather), we observe two populations of ratings: ratings associated with that condition only (e.g., sunny weather), and ratings associated with any other condition of the same contextual factor (e.g.,rainy, cloudy, snowy and stormy). We use the Wilcoxon rank-sum test to compare these two populations. More speci cally, we test the null hypothesis that the ratings from these two populations are sampled from a continuous distributions with equal medians. If we reject the null hypothesis, the medians are di erent, which means that the users tend to rate items di erently during the tested condition (e.g., sunny) compared to the other conditions (e.g. rainy, cloudy, snowy and stormy). If this is the case, we determine that the tested contextual condition is relevant. Otherwise we determine that since there is no di erence in ratings, such condition has no impact and is thus irrelevant. The Wilcoxon rank-sum test was chosen over the t-test because the compared samples were not normally distributed.
The described approach was done on the population level, i.e., on the data from the whole population and not for each user separately. Hence, contextual conditions are detected as relevant or irrelevant with regards the whole population
Once the irrelevant conditions of each contextual factor are detected, we proceed to merge them with relevant categories. In order to determine which categories should be merged, we compare the distribution of ratings for each irrelevant condition with the distribution for each relevant condition separately (e.g. sunny vs. rainy, sunny vs. cloudy, sunny vs. snowy and sunny vs. stormy). Once again, this is tested with the Wilcoxon rank-sum test. In this case, if the test determined that the medians of the ratings distributions for the irrelevant and relevant conditions are equal, we determine that these conditions can be merged. This is because there is no di erence in rating when users were in these two separate conditions.
However, the proposed methodology might yield a type of error during merges. It might occur that we determine the two conditions could be merged when in fact they should not. This exception might occur if the distributions were similar, yet, for di erent user-item pairs ratings were drastically di erent on di erent contextual conditions. This is an open issue we plan to address in the future work. 2.4
Once we determine which contextual condition should be merged we implement merging in two separate tasks: (i) improving the questionnaire and (ii) improving the model. 2.4.1
In our system we acquire the contextual information explicitly through questionnaire. Hence, we implement merging in the questionnaire by modifying the list of possible contextual conditions users choose from. For example, in our system we have the contextual factor season, which contains the following contextual conditions: spring, summer, autumn, winter. Let us say that we have determined summer to be an irrelevant condition and that it should be merged with the relevant condition autumn. We would simply change possible answers in the questionnaire into: spring, summer/autumn, winter. In this way we lower the amount of possible answers, and stop associating ratings with irrelevant contextual condition. Of course, if possible, a new name for the combined condition could be used in the questionnaire.
If contextual information would be acquired implicitly through sensors, merging would be implemented in the step of processing sensor data into contextual conditions. 2.4.2
In this study we used the contextualized matrix
factorization algorithm for modeling the interaction between the
users and the movie items. Matrix factorization (MF) is a
latent-factor model that is widely used in RS ([
The contextualized users' biases with the matrix factorization (CUB-MF) approach uses the contextual information for the contextualized users' biases. Only the users' biases are context dependent. This approach follows the idea that the users' rating behaviour is di erent on di erent occasions.
The matrix factorization in CUB-MF was made using the following equation: r^ (u; h) = + bh + bu(c) + q~hT p~u; (1) where r^ (u; h) is the predicted rating for user u and item h, q~h is the item's latent-feature vector, p~u is the user's latent-feature vector. The user's bias bu and the item's bias bh measure the deviations of the user's u and the item's h ratings from the rating average .
To inspect the impact of merging contextual conditions of contextual factors on the rating prediction, we trained our model for each contextual factor separately, i.e. using only a single contextual factor at the time.
The standard way, of training (without merging) the contextualized model is done in the following way: the algorithm loops through all the ratings in the training set, and calculates the prediction error e(u; h; c) = r(u; h; c) r^(u; h; c) for each predicted rating r^(u; h; c) and real rating r(u; h; c), for user u, item h and contextual condition c. Among other uncontextualized parameters, we modify the contextualized user's u bias by the equation: bu(c) bu(c) + (e(u; h; c) bu(c)): (2)
Hence, if the contextual condition was, for example summer, we would update bu(sunny).
When we implement merging during modeling, for each calculated error of prediction, we update the contextualized parameters of all merged conditions, if such exist. Therefore, if, for example, the contextual condition summer has to be merged with the condition autumn, we would use e(u; i; summer) to update bu(sunny) and bu(autumn) simultaneously. In this way we reduce the negative impact of sparsity by utilizing ratings associated with irrelevant conditions to train parameters contextualized by the relevant ones. In addition, during training, for each calculated error of prediction, we also train the uncontextualized users' biases. Once the model is trained, on the testing set, the uncontextualized users' biases are used to predict the ratings associated with the irrelevant contextual conditions. In this way, the algorithm simply avoids the contextualized rating prediction in the case of the irrelevant contextual condition. 2.5
In order to test the positive impact of our procedure for detecting irrelevant contextual conditions, and determining merges, it is important to compare the results from our approach with the fair baseline. It could be that the improvement in the rating prediction is not due to our merging technique, but due to any type of merging simply because we lower the sparsity. In another words, it is important to test if we would get equally improved results by randomly merging several conditions.
Therefore we have implemented a random merging method in the following way: for every contextual factor we count the exact number of irrelevant conditions and determined merges, and select the same amount of random conditions and random merges. In this way we replicate the same amount of merges but select the conditions to be merged randomly.
The results for our approach and the random merges are achieved on 10 di erent folds.
In the cases of the time, daytype and location contextual factors, all conditions were found irrelevant, hence no merges are possible. In the cases of the decision, interaction, and physical contextual factors, all conditions were found relevant, hence no merges are needed. For the remaining contextual factors, table 2 contains the results of the contextualcondition-relevancy detection, and merges determination.
The gures 2, 3, 4, 5, 6 and 7 show the results of the matrix factorization rating prediction.
On each gure, boxplots are presented: one from our merging method (merge) and the second one from the random merge baseline (randMerge). Both boxplots represent the RMSE di erence between the basic model without merging (basic), and the merge and randMerge approaches. Therefore, if the result is above zero, the merging approach performed better (lower RMSE) than the basic approach without merging.
The Wilcoxon signed-rank test was used to test the statistical signi cance of the di erences between basic and merging approaches. If the di erence was statistically signi cant the box plot is colored green, otherwise it is colored red.
In the previous section we could observe di erent results
for di erent contextual factors. It is interesting to note that
contextual factors for which all the contextual conditions
were detected as irrelevant (time, daytype and location) are
those that were detected irrelevant themselves in our
previous work [
Implementing merges in questionnaire can be easily achieved for season, weather and mood, by simply merging conditions between possible answers. However, for social contextual condition, as it is shown in Table 2, there are con icts which prevent us for merging. For example, the condition parents can be merged with alone and family, but not with partner, as it is the case with the conditions friends, colleagues and public.
Furthermore, for end emotion and dominant emotion, the irrelevant condition fear can be merged with multiple conditions (sad, happy, surprised ), however each of them is relevant and should be used alone as it is. Therefore, an opened issue remains - how such cases should be handled in questionnaires.
By implementing the proposed procedure for the detection of irrelevant contextual categories, and the proposed way to manage merges during modeling, we achieved significantly better results than without merging for the contextual factors weather, social, end emotion and mood. For the contextual factor season we achieved an improvement, however it was not statistically signi cant (Figure 5). In each case our procedure outperformed random-merging baseline, which did not lead to signi cantly improved results in any case. However, even in the case of random merging there is tendency towards better results with fewer conditions which con rms our assumption from the introduction: many contextual conditions have a large impact on the sparsity of ratings in the contextualized models.
The only contextual factor for which we observed unexpected results is the dominant emotion. In this case we achieved signi cantly worse results for both our approach and the random-merging baseline. We believe that this is an interesting open issue that we plan to address further in the future.
In this paper we proposed a procedure for detecting the relevancy of contextual conditions and how to manage such conditions by merging them with relevant ones. We implemented merging of contextual conditions on the questionnaire for acquiring contextual data, and into contextualized modeling based on matrix factorization. The results showed signi cantly improved results by our method, except in the case of one speci c contextual factor.
For the future work we plan on researching further why anomalies can occur and how to predict and avoid them. Also, we are interested in solving con icts described in this paper regarding the implementation of merges in questionnaires. 5.