The cold start problem describes the initial phase of a collaborative recommender where the quality of recommendation is low due to an insuficient number of ratings. Overcoming this is crucial because the system's adoption will be impeded by low recommendation quality. In this paper, we propose capturing context via computer vision to improve recommender systems in the cold start phase. Computer vision algorithms can derive stereotypes such as gender or age, but also the user's emotions without explicit interaction. We present an approach based on the statistical framework of bandit algorithms to incorporate stereotypic information and afective reactions into the recommendation. In a preliminary evaluation in a lab study with 21 participants, we already observe an improvement of the number of positive ratings. Furthermore, we report additional findings of experimenting with afective computing for recommender systems.
Recommender systems (RS) match items to users, therefore the
accuracy of recommendations is highly dependent on the quality
of information the system has about these. Collaborative
filtering (CF) has frequently been used if the items’ characteristics are
unknown or it is costly to derive them. CF systems are, however,
not suited for scenarios where the user is anonymous and interacts
with the RS only for a short period. For example, a smart display
inside a fashion store could provide recommendations, however,
the interaction will be brief and tentative. In such cold start
scenarios, literature suggests including context and stereotypes into
the recommendations [
In the following section, we describe the foundations of our RS: bandit strategies and facial classification using computer vision. 2
Ever since Grundy [
In real-world applications, recommendations are often linked to a reward. For example, the purpose of recommendations in a shop is to improve revenue by suggesting products to customers that they are more likely to buy. However, calculating the probabilities of a successful recommendation directly is usually not possible due to a lack of information about the customer’s taste and the attractiveness of items.
Bandit strategies provide a computational framework that trades
of profit-maximization via items that are known to sell well and
experimentation with items whose potential is yet to be determined.
The terminology stems from the probability theory of gambling [
Our model extends the approach of Bounefouf et al. [
U P = argmax(sim(U t , U c ))
U c ∈P Our adapted similarity metric is the weighted sum of the similarity metrics for age, gender, and EF, the combination of emotions and feedback. α , β, γ are weights associated with these metrics, defined in the following subsection: sim(U t , U c ) = α · sim(at , ac ) + β · sim(дt , дc ) + γ · EF (2)
EF , short for emotional feedback, corresponds to the sum of k
afective reactions simk (ekt , ekc ) ∈ [
In our approach, we propose using facial classification through the use of computer vision to infer age, gender and emotions as contextual information within a contextual bandit algorithm. 2.2
Computer vision has already been used to improve systems situated
in public places. For example, Müller et al. [
The accuracy of classification and the runtime performance of
computer vision algorithms have improved over the past years and
with YOLO [
In our RS, the items are displayed to the user successively. While the user inspects the items, she is observed by a camera whose imagery is continuously analyzed by computer vision. In this section, we first present how we incorporated computer vision into the recommendation task, followed by the experimental setup and our findings. 1https://azure.microsoft.com/en-us/services/cognitive-services/face/ 2https://www.kairos.com/emotion-analysis-api 3https://www.afectiva.com/product/emotion-sdk/ (1) (3) EF = Õ k simk (fkt , fkc ) · 1 + simk (ekt , ekc ) 2i Task 2: Let M be the set of items, Mt the items seen by the current user U t and MP ∈ {M \Mt } the items recommended to the user U P , but not to U t . After retrieving MP , the system displays the next item m ∈ MP to U t while observing the user’s afective reactions during presentation.
Task 3: After receiving the user’s reward, the algorithm refines its item selection strategy with the new observation: user U P gives item mP a binary reward. The expected reward for an item is the average reward per total number of ratings n.
Our adapted Contextual-ε-Greedy recommends items as follows:
argmax (expectedReward(m)) if q > ε
m = MP (4)
random ((M \ Mt )) otherwise
In Equation 4, the random variable q is responsible for the
exploration versus exploitation behavior. In our approach it is uniformly
distributed over [
To influence the original ε-Greedy algorithm with contextual information, ε is computed by maximizing Equation 2, the similarity of the current user’s profile U t to the profile U P of the most similar other user: ε = 1 − argmax(sim(U t , U c )) (5) The Contextual-ε-Greedy strategy is driven by the stereotypic similarity of the current user to previously seen users. In this first experiment, we used α = β = 0.25 and γ = 0.5 as weights for Equation 2.
Gender similarity is binary, due to output of the employed facial classification algorithm. Either it matches, or it does not: sim(дt , дc ) ∈ {0, 1}.
Age similarity is more fuzzy and we have not found an
established similarity measure in literature. Therefore, we constructed
an ad-hoc similarity measure sim(at , ac ) ∈ [
Emotional similarity measures the afective response to a displayed item in comparison to the emotional reaction of previous users to it. As previously mentioned, today’s computer vision algorithms are capable of detecting several emotions at once. Therefore, it is calculated by the cosine similarity of two emotion vectors, as can be seen in Equation 6.
sim(et , ec ) = n Õ e¯it · e¯i
c i=1 vt n Õ i=1 (e¯it )2 vt n Õ i=1 (e¯ic )2 3.2
Microsoft Face analyzes the user’s face for age, gender, and up to eight emotions. Experimenting with the computer vision service before the main experiment showed that users tend to express their emotional reactions shortly before requesting the next item and maintain their facial expression for some time when the next item is already shown. We call this ‘overflowing emotions’ , as the user’s emotional reaction to the previous item overflows to the current item and is then adjusted during the consumption and exit stage. Since we are interested in the actual response to the item after the content has been processed, we used the following weighted average over all analyzed frames n as the aggregated metric to emphasize the emotions from the exit stage.
n Õ 2i · ei e¯ = i=1 n Õ 2i i=1
Figure 1 shows the comparison of the mean value to our proposed weighted average. Over the course of three items, the level of observed happiness is shown in orange for 15 frames in the case of Item A. Since we assume that the important reaction to the content is at the end of the item display period, we are quite satisfied with our weighted mean calculation. Note that we used a sampling rate of one analyzed frame per second.
An alternative would have been to aggregate over the last p% of the frames. While we think that our measure is more robust, an in-depth analysis of diferent aggregation strategies is left for future work. Another idea for separating successive content is to show a neutral screen for some time before showing the next. It is, however, unclear what an adequate time is for that, as users tend to show emotions for an unknown duration and may find this delay annoying. To evaluate our approach, we implemented an image recommender prototype using Python. Figure 2 shows the high-level architecture: the core part is a Flask4 web server that serves web pages with the recommendations based on context information (age, gender, emotions) from the computer vision service and the history of user interactions retrieved from a PostgresSQL5 database.
To answer our second research question, we compare our variant of the Contextual-ε-Greedy with the traditional ε-Greedy in a controlled lab experiment. The experimental procedure was the following: The participant’s task is to rate images. Hoping to evoke a large spectrum of emotions, we used a self-scraped data set of 3000 memes from the social web platform 9gag6 over the period from January 24, to February 9, 2018. The subject is instructed to take a seat in front of the screen with a webcam, it pointed out that the camera is recording and information is being stored according to local data privacy protection laws. She is asked to view consecutively displayed images and provide feedback for each one in the form of a ’like’ or ’dislike’ rating. The recommendation engine attempts to optimize the amount of positive feedback using our Contextual-ε-Greedy or the baseline ε-Greedy. Each subject is shown 60 images per strategy, which is our independent variable. The order of strategy is selected at random without the subject being aware of this.
We conducted the experiment in April 2018 in Garching with 21 volunteers (11 f / 10 m) afiliated with the Technical University of Munich. The subjects’ ages varied between 19 to 31 years with a mean value of 24.09. The dependent variables are the users’ feedback 4http://flask.pocoo.org 5https://www.postgresql.org 6https://9gag.com to the item, the detected afective cues from the computer vision service and additional information collected with a questionnaire.
In the convergence analysis of the algorithms, we observe an improvement of the accuracy of time, i.e., the number of positive ratings, in both recommendation strategies. To showcase this, we ift a linear model over the algorithm convergence described in Table 1. Over the course of 21 observations, the Contextual-ε-Greedy starts slightly worse with 46.64% positive rewards; however, it improves faster over time reaching 60.7% at the end of the experiment. Note that the diference between the strategies is not significant and this model should not be used to predict further observations. Clearly, 21 observations with 60 ratings each are not enough for the bandit algorithms to converge.
A closer look into the properties of the Contextual-ε-Greedy algorithm reveals avenues for improvement. Figure 3 depicts the similarity of a participant’s stereotypic attributes to the previous subjects. The most similar user pair per column has the lowest ε and was leveraged by the Contextual algorithm for recommending the next item (cf. Equation 4). A clearly visible pattern is that the same gender plays a dominant role in the distance measure. Depending on the recommended items, this could be adjusted in future studies.
Further, we notice that the Microsoft Face algorithm mostly detected two emotions. Overall, happiness and neutral make up 93.65% of the observed emotions, with neutral being the dominant emotion. However, as seen in Table 2, positive feedback is more likely if the afective response was happiness instead of neutral.
Overall, the subjects rated 53.97% of the items positively, although this varied a lot per user, ranging from only 3 positive ratings up to 47 of 60. Also, the experiment showed that the duration of item consumption varies, underlining the need for a dynamic aggregation of the analyzed frame as in Equation 7.
Bandit algorithms provide a robust framework not only for online advertisement, but also for personalized recommendations. The possibility of calibrating the exploration vs. exploitation probabilities using weighted similarity measures is an elegant way for the hybridization of recommendation and active learning. Although computer vision has not yet reached its full potential, it is suficiently afordable and accurate to experiment with for RS research.
In this paper, we have presented an approach for recommending images using bandit algorithms and computer vision focusing on improving recommendations in the cold start phase. Although our contextual bandit algorithm was not significantly better than the baseline, our work comprises the following contributions: (1) We have developed a practical approach for using information from facial classification within RSs, (2) we presented an adaptation of the Contextual-ε-Greedy suited for incorporating stereotypic information, (3) we developed a strategy with a weighted average to mitigate the overflowing emotions problem, and (4) we have shown using a lab study that by putting the pieces together, an improvement of the recommendation accuracy could be achieved. While this study was conducted with the informed consent of the participants, the unconscious measuring of people’s emotions in real-world applications is critical with respect to privacy concerns.
Having realized this prototype based on many assumptions, we can highlight the path for further research: Our post-mortem analysis has shown the necessity of an evidence-based method for adjusting the weights of the hybrid similarity measure. Having identified the ‘overflowing emotions’ problem in sequential recommendations, an in-depth analysis thereof would be interesting. Finally, we plan to analyze the long term convergence of our bandit recommender algorithm in a larger field experiment against simpler baselines, e.g., random items, and to investigate the accuracy of emotional classification and its potential impact on performance.