In this article we propose an interactive Web-based movie recommendation system namely MISRec employing object recognition for movie thumbnails. The proposed system carries out object recognition on movie thumbnails or DVD cover-photos in a real-time manner, and recommends movies based on user's historical preferences and the query intention. Unlike typical preference-based recommendation systems which provide recommendations merely by using the user's historical preference, MISRec provides a user with a choice to specify her search intention in a user-friendly and interactive manner and to alter/adjust the recommendation results which in turn can lead to increased utility of recommendation and user satisfaction.
Imagine it is the weekend and you are sitting alone or with your partner at home and plan to choose a right movie to watch. Among the many available choices e.g. provided by an online Video on Demand (VoD) service such as Net ix, it is not surprising if you soon nd yourself overwhelmed with numerous possibilities in the movie catalog. Recommender systems (RSs) are subclass of information ltering and search techniques which ease this task by providing recommendations about unseen items. However, what happens if the movies recommended by the Net ix RS are not the type of the movie the user is expecting to watch?
Preference-based recommender systems typically provide recommendations
by predicting a user's preference on unseen items based on her historical
preferences without giving her the opportunity to alter or modify the recommendation
results. Although such form of passively receiving recommendations can assist
a user with discovery of new items, it can also refrain her of actively inputting
her interests to obtain di erent recommendations as in reality, users can have
multiple watching interests or search intentions [
Based on these situations, in this paper, we propose an interactive
Webbased movie recommendation system namely MISRec which carries out object
recognition on movie thumbnails or DVD cover-photos in a real-time manner.
By pointing the DVD cover-photo toward the Webcam of MISRec, a user can
receive a recommendation based on her past preferences and the current search
query intention. We designed and implemented the system to be used easily and
intuitively during shopping at media stores or at home [
For the purpose of our study, we have developed MISRec, a web-centric testing framework for the movie domain which can be easily con gured to facilitate the execution of controlled empirical studies. MISRec contains a catalog of movies containing about 12K movies where users can browse, retrieve the detailed description of each item, rate them, and receive recommendations. In this subsection, we describe the ow of how to use the proposed system from initial preference elicitation during sign-up process, the product recognition until the user receiving the nal recommendations
Step 1: The interaction begins with a sign-up process, where a user is initially asked to provide his/her personal information: age, gender, nationality, some optional social media data and an indication of number of movies watched per month. This step is shown in Figure 1a.
Steps 2: Afterward, the user is invited to browse the movie catalog and select 5 movies from her favorite genre which she can do by scrolling through movies' years of production in a relatively easy manner. These steps are shown in Figure 1b and 1c .
Steps 3: The user is then invited watch the trailer of the selected movies online and rate them using a 1-5 point scale (1 = low interest for/appreciation of the movie; 5 = high interest for/appreciation of the movie). This step is shown in Figure 1d and 1e.
On the basis of these ratings, a pro le is built for the user based on 3 attributes
of the movie (i) the genre (ii) the style [
Step 4: In this stage, the user is invited to show the image of her interest
movie to the system. The user can do this by showing the movie thumbnail
to the system Webcam easily through her cellphone or (if in a media store)
by showing the DVD cover-photo of the movie. The system is engaging in
order to motivate users to use the system and provide feedback as shown in
Figure 1g and 1h.
3 The work is partially covered by a US patent application under the title of
\Enhanced content based multimedia recommendation method", with application
number 15/277490 http://www.polimi.it/index.php?id=6247&sel_brevetto=5093
(a)
(d)
(g)
(b)
(e)
(h)
(c)
(f)
(i)
On MISRec backend works a product recognition system developed by Telecom
Italia (TIM) industrial partner, which is trained by thousands of movies cover
photos obtained from IMDB and that can recognize an input query image based
on its visual similarity to the training covers dataset. This module implements
the MPEG Compact Descriptors for Visual Search (CDVS) standard and
perfectly ts the performance requirements of these kind of scenarios [
Motivated by [
Our goal is to improve the state-of-the-art of multi-media recommender
systems by designing user-engaging interaction paradigms. We plan to conduct a
user study in order to measure e ectiveness of the proposed system. We hope
that our interactive cover-photo recognition systems can pave the path for
design of other user-friendly interactive scenarios in the design of next
generation RSs. Some examples include: object recognition on kids' toys in order to
recommend their favorite cartoons/programmes, gesture-based interaction and
emotion recognition using user's facial expressions [
This work has been partially funded by Telecom Italia S.p.A., Services Innovation Department, Joint Open Lab S-Cube, Milano.