=Paper=
{{Paper
|id=Vol-1705/02-paper
|storemode=property
|title=Interactive Recommending: Framework, State of Research and Future Challenges
|pdfUrl=https://ceur-ws.org/Vol-1705/02-paper.pdf
|volume=Vol-1705
|authors=Benedikt Loepp,Catalin-Mihai Barbu,Jügen Ziegler
|dblpUrl=https://dblp.org/rec/conf/eics/LoeppB016
}}
==Interactive Recommending: Framework, State of Research and Future Challenges==
https://ceur-ws.org/Vol-1705/02-paper.pdf
Interactive Recommending:
Framework, State of Research and
Future Challenges
Abstract
Benedikt Loepp In this paper, we present a framework describing the var-
University of Duisburg-Essen
ious aspects of recommender systems that can serve for
Duisburg, Germany
empowering users by giving them more interactive con-
benedikt.loepp@uni-due.de
trol and transparency in the recommendation process.
While conventional recommenders mostly operate like black
boxes that cannot be influenced by the user, we identify
Catalin-Mihai Barbu four aspects properly connected with the recommendation
University of Duisburg-Essen algorithm-namely input data, user model, external context
Duisburg, Germany
model and presentation-as essential points in which a sys-
catalin.barbu@uni-due.de
tem may be enhanced by additional interaction possibilities.
In light of this framework, we take a closer look at prior and
present solutions to integrate recommender systems with
Jügen Ziegler more inter-activity and describe future research challenges.
University of Duisburg-Essen Regarding these challenges, we especially focus on expe-
Duisburg, Germany
riences gained in our own work and outline future research
juergen.ziegler@uni-due.de
we have planned in the area of interactive recommending.
Author Keywords
Recommender Systems; Interactive Recommending; Mod-
els, User Experience; User Interfaces; Survey.
ACM Classification Keywords
Copyright is held by the author/owner(s). H.3.3 [Information Storage and Retrieval: Information Search
EICS’16, June 21-24, 2016, Bruxelles, Belgium. and Retrieval]: information filtering, search process; H.5.2
[Information Interfaces and Presentation: User Interfaces]:
evaluation/methodology, graphical user interfaces (GUI),
user-centered.
3
Introduction pects to the framework we propose in this paper, the focus
Providing users with interactive control over the recommen- of the authors lies on visualizations and related aspects.
dation process has only recently started to receive more How to offer users more control at the different stages in
attention in Recommender Systems (RS) research [25, 26, the recommendation process is only one of many aspects
40]. In terms of objective error metrics, recommender al- mentioned.
gorithms are already quite mature and only small improve-
ments can be expected from further optimizing algorithmic In this paper, we will therefore provide a closer look at this
precision [40]. However, high accuracy is not the only factor particular issue: First, we present a framework of interac-
determining user satisfaction [26]. It is increasingly recog- tion in RS that describes the range of possibilities users
nized that user-related aspects such as control, trust and have for influencing the recommendation process. Next,
transparency influence the users’ perception of the recom- we provide a detailed overview of the four aspects we have
mendations even more, and may contribute considerably identified around the recommendation algorithm itself that
to higher satisfaction [26, 40]. This makes it an important allow for integrating additional interaction-input data, user
research goal to let users influence the recommendation model, external context model and presentation. We survey
process and to make it more comprehensible [25, 26, 40]. some of the most influential work related to each aspect,
derive future research challenges, and outline solutions to
Several models exist that describe typical user behavior deal with them that are especially promising from our point
during the recommendation process. In earlier work [31], for of view and subject of our upcoming work. Finally, we con-
instance, we have proposed a model comprising three inter- clude the paper with a short summary and discussion.
action loops, which represent a) the user’s interaction with
the recommendations themselves, b) selection and weight- A framework for interactive recommending
ing of properties related to the recommended items, and Figure 1 shows our proposed framework: Blue boxes repre-
c) adaptation of entire recommender applications. Various sent components containing data, models, or presentation
models have also been introduced in the area of informa- that may be manipulated by the user to adapt the system’s
tion retrieval, particularly aiming at examining the users’ outcome according to his or her current needs. The cen-
information-seeking behavior [28, 34]. Due to their focus tral recommender algorithm(s) (red circle) that process in-
on document collections and explicit search tasks, these put data and models may also be interactively influenced,
models are however not directly applicable to RS. On the for ex-ample, by changing an algorithm’s parameters or by
other hand, models in the area of RS research often focus rear-ranging the processing steps in the case of hybrid sys-
on conversational and critique-based systems [44, 8], more tems.
basic feed-back processes [41], or describe system usage
distinguished by different feedback types [22], i.e. ways to All of these components can be considered important with
elicit implicit or explicit rating data. In [9], the area of inter- regard to user-perceived quality [25, 26, 40], e.g. perceived
active RS is surveyed by means of a basic model compris- recommendation quality or transparency of the results.
ing those recommender components that can be extended There have indeed been efforts to allow users to manip-
to allow for additional interaction. While similar in some as- ulate the recommender algorithms themselves [20], to
4
Figure 1: Framework for interactive recommending delineating the points in the recommendation process where users can be provided with
additional means for interaction.
5
choose from different algorithms [14], or to change their in- Collaborative Filtering (CF), the most frequently used rec-
fluence in hybrid settings [6, 30]. However, in the following, ommender technique [42], relies on input data usually lim-
we concentrate on a) input data related to users or items ited to user feedback, which is either explicitly provided
provided for the recommender, b) the user model inferred through ratings or implicitly observed based on behavioral
from, e.g., the user’s preferences, needs, and emotions, c) data [22]. Other methods use tags [43] or rely on a social
the external context model representing the user’s current graph, i.e. relationships between users [17, 18]. Particu-
situation, i.e. his or her environment, used device, etc., as larly in content-based filtering [11], item attributes or other
well as d) the presentation of the recommender’s results. content- related information are used to recommend items.
For each aspect, a (nonexhaustive) list of properties is pre- However, in all cases, user or item data primarily serve as
sented which may characterize the respective part of the input for the algorithms that generate recommendations.
system. Arrows (orange) visualize the process flow starting Only few methods exploit, for instance, tags [12, 46] or item
from possible preprocessing steps and selection of appro- attributes [30] to let users select and weight certain prod-
priate input data for the algorithms, which then generate uct characteristics, or visualize social connections [17] to
the recommendations, i.e. adapt the presented result set. improve users’ understanding of the recommendation pro-
Therefore, the algorithms are able to exploit user model and cess.
external context model, which in turn may be inferred by
means of the users’ feedback or are generally affected by Eliciting user preferences is an important step in order
their interaction with the system. to obtain the input data necessary for the employed al-
gorithms, which is especially relevant in cold-start situa-
Current position and future work tions. Various methods have been proposed to overcome
Although much effort has been put into improving the al- the problems of traditional rating-based interfaces. Prior
gorithms used in RS, other aspects still lack attention from research has shown that ratings may be inaccurate [2]
the research community, especially regarding their role in and that users prefer com-paring items instead of rating
increasing the recommenders’ transparency and the users’ them [23]. In general, different users seem to benefit from
influence on the systems [25, 26, 40]. In the following, we different interaction possibilities [24]. Thus, we among
therefore have a closer look at the four relevant aspects others have proposed alternative preference elicitation
from our model, related work and future challenges. methods: Our choice-based approach [32] allows users to
state their initial preferences without the need to rate items.
Input Data When compared to a conventional rating pro-cess, it has
The input for a RS, i.e. user or item data, is not only used been shown to be more beneficial in terms of, e.g., per-
by machine learning techniques to generate recommen- ceived effort, control, and subjective recommendation qual-
dations, but also represents an important part of such a ity [32]. Other authors have also experimented with novel
system that might be exploited to let users influence the ways to elicit preferences, for example, by letting users pick
recommendation process and to improve their understand- from groups of items [7] or by mapping their choice of cer-
ing of why certain items are recommended. tain pictures to factors describing their preferences [37].
6
We argue that exploiting input data for purposes other vances as well as by considering additional and multiple
than feeding them into the algorithms can be an important data sources [27]. However, we argue that an adequate
means for giving users more control over the recommen- user model should not serve only as input for the algo-
dation pro-cess. A possible challenge for future research rithms, but might also be exploited to let users adapt the
can therefore be seen in developing techniques that create system’s output and to increase their understanding of the
new ways of interacting with user or item data. This may recommendation process.
comprise filtering these data even before applying the algo-
rithms or visualizing them in order to improve the user’s un- Indeed, user preferences can be modeled based on other
derstanding of product space and his or her position inside in-puts than item ratings. In principle, all forms of implicit or
it (as it has been done, for instance, through maps show- explicit feedback [22], also given for item-tags [43], content-
ing a “recommendation landscape” [16]). By building on the related properties, etc., can be considered. In content-
aforementioned works, we particularly want to improve pref- based filtering, user models are typically learned by prob-
erence elicitation for CF: Providing alternatives to simply abilistic methods or nearest neighbor algorithms based
rating a set of items seems to be a promising way to alle- on what products the user has bought, liked or viewed be-
viate the cold-start problem [32, 37, 7, 13]. Now imagine fore [11]. Even psychological aspects such as emotions or
an extension of [32] that provides users with comparisons personality can be taken into account [39]. However, none
that not directly feature the items (presented in form of, e.g., of these approaches has been developed with the specific
movie posters, hotel descriptions or metadata of cameras), goal of improving interactivity. In contrast, the only way to
but enables them to get an experiential impression of the influence the results and to (implicitly) refine the user model
products. Specifically, a system could instead use compo- is typically by giving some kind of relevance feedback [11].
sitions of pivotal scenes captured from the movies, photos In social RS, it has been shown that enabling the user to
of the hotels and their amenities, or images actually taken adjust the importance of the mentors used for rating predic-
with the respective cameras. Thus, users would be able to tion increases transparency and satisfaction [17]. But, this
express their taste towards more general characteristics is one of the only very few examples that already provide
than just towards individual products (they may find hard to some insights in the model by means of visualizations and
assess or do not know about). at the same time exploit it to allow the user actively influenc-
ing the process.
User Model
The quality of the user model, typically learned by means Existing interactive RS, e.g. [6, 8, 46], are often developed
of the user’s feedback provided during interaction with independently of model-based CF, and thus cannot bene-
the system, is a critical determinant for the accuracy of fit from the availability of models inferred by these efficient
today’s recommender algorithms. Model-based CF [42] and accurate techniques. MF algorithms result in latent fac-
techniques such as Matrix Factorization (MF) [27] are very tor models where each user is individually represented by
prominent examples that use ratings provided by users to a vector whose entries describe how much the user is in-
efficiently generate precise recommendations. The respec- terested in the respective factors [27]. While it cannot be
tive methods have been improved both by algorithmic ad- expected that improving the algorithms will further increase
7
the actual user satisfaction with the systems [26, 40], la- to manipulate the latent user model by means of easy-
tent factor models may also be used for other purposes to-understand tags. This seems especially useful in cold-
than generating precise recommendations. For instance, start situations, because selecting a small number of tags
they already have served to visualize an item landscape leads to a meaningful new user profile without requiring
by reducing the high-dimensional factor space to a two- the user to rate items first. Besides, as the abstract models
dimensional map [16]. Beyond that, the information used are mostly opaque, hindering the user to understand the
to model the current user’s individual interests, i.e. his or learned profile and hence the generated recommendations,
her own user vector, may be exploited in even more differ- one can imagine using the introduced semantics to better
ent ways. In [38], for example, the characteristics of an item explain the user model.
have been visualized by means of latent factors. Applying
the proposed method to users instead could result in so- Overall, while the aforementioned approaches already intro-
called 2D feature maps showing named regions that the duce more control over the user model, many more aspects
current user is interested in. However, the only chance for make this part of a RS particularly interesting for increas-
users to affect their preference profile in model-based CF is ing the level of interactivity. For example, privacy concerns
usually through explicit feedback given by further ratings. In suggest that users should be able to select themselves the
light of this fact, it is therefore-from our point of view-a major information that will be stored in the user model and subse-
challenge to improve these systems significantly by letting quently exploited for generating recommendations. Since
users actively adjust the user model. mediating user models, i.e. importing and integrating them
from other systems [4], seems promising for increasing ac-
First attempts allow users to manipulate their user vector curacy and providing cross-domain recommendations, this
by other means than just rating items, i.e. more directly. should also be considered as an important subject when
With the choice-based approach mentioned before [32], it is trying to bring more interactivity and transparency into a
possible to navigate through the factor space to generate a RS.
model representing the user’s situational interests. Extend-
ing the landscape approach of [16] to 3D, the map’s altitude External Context Model
can be used to reflect the user’s preferences (mountains Regarding long-term interests, RS are already able to suf-
represent areas of interest while valleys indicate low rel- ficiently derive the user’s preferences, learn an adequate
evance) [29]. In addition, the user is able to reshape the user model, and present him or her with well-fitting rec-
landscape in order to manipulate the user vector, thus lead- ommendations [26, 40, 42]. However, the user’s context,
ing to new results. We have also investigated other ways i.e. date and time, season, weather, location, company of
to import semantics into the abstract latent factor space, other people, used device, and many other aspects that
particularly by associating user-provided information such depend on the user’s current situation are often not con-
as tags with the factors [12]. While this was already known sidered in the recommendation process, although a num-
to be effective in terms of objective accuracy [27], we have ber of context-aware recommending approaches has been
confirmed this finding also with respect to subjective qual- proposed in recent years [1]. In fact, many systems do not
ity [13]. Moreover, our approach introduces a novel way even distinguish between long-term and short-term prefer-
8
ences, and especially disregard that the latter are strongly letting the user actively adjust these factors is thus typically
coupled with context [15]. not possible-although it would give him or her the control
which kind of information, e.g. about restaurants (nearby
A typical example is that a user might be interested in dif- and open vs. more general), is actually desired. In [3], con-
ferent things depending on, e.g., the currently used device: textual information is used to explain recommendations,
When using a smartphone on the go, he or she potentially for instance, by stating that a location is especially worth a
wants suggestions for open restaurants nearby, while in- visit at a specific time of the day. In addition, the proposed
formation that is more general would be appropriate when system is one of the few exceptions that allows the user to
sit-ting in front of a desktop PC. Such variables indicated influence which contextual factors to consider in the rec-
by the user’s external context have already been taken into ommendation process, although this is limited to switching
account, resulting in, among others, restaurant and travel them on or off. Thus, finding new ways of integrating this
recommenders, music recommenders specialized for dif- part of a RS with interactive control seems to be a particular
ferent purposes (in the car, at the gym, for groups, etc.), or fruitful area of future research.
news RS [1]. The advent of smartphones has increased
the research community’s interest in developing “mobile” Presentation
context-aware recommenders even more. However, al- The presentation of recommended items has also received
though it would be particularly useful due to their increased relatively little attention by comparison. Aspects such as
complexity and since more information, i.e. context, has to what information to present, how to present it, when and
be considered, context-aware RS often lack richer interac- how often to present it, and how much of it to present for
tion possibilities [1]. any given recommendation are important when discussing
inter-activity in RS. Prior work has explored the persuasive-
So far, most work has been done on the algorithmic side, ness of different types of recommendation lists and combi-
either by specializing existing methods to also integrate nations of text with images [36]. Other researchers studied
context or by developing techniques specifically for that use different approaches to visualize the results [45], suggested
case. More details on how to incorporate contextual infor- a model for timing recommendations [10], or determined
mation may be found in [1]. However, only little attention the number of results that leads to high choice satisfaction
has been paid to increasing user control in context-aware without increasing choice difficulty [5]. However, most of
recommenders [9]. Some conversational systems adapt this work stops short of considering interactivity a major fac-
their dialogues implicitly based on the user’s interaction tor. Consequently, ways to increase user interaction at this
sequences [33]. Similarly, changes in the user’s interests stage of the recommendation process remain rather unex-
can be captured to fit the results [19]. Based on the user’s plored.
feedback, not only the user model, but also contextual fac-
tors can be re-fined, e.g., to filter out those restaurants that Our work takes into consideration the recently made ar-
do not suit the current situation [1]. Yet overall, existing re- gument that novel approaches in RS can also stem from
search often tries to derive the required contextual infor- under-standing how people make choices [21]. There-
mation automatically [1]. While this indeed has its benefits, fore, we aim to investigate choice support strategies that
9
are not typically related to recommendation technologies, The presentation of results could also be improved by using
such as “combine and compute” (i.e. derive relationships social media data: By mining users’ past bookings as well
from available data to show more relevant information) as their reviews, a complex network consisting of users, ho-
and “design the domain” (i.e. adapt the interface to facili- tels, and hotel attributes can be created. This would allow
tate choice) [21]. As an ex-ample, consider tourists look- identifying with greater accuracy items a user is likely to find
ing for a hotel room on a booking website. Based on the at-tractive based on the attributes mentioned in his or her
choices they make during their search-destination, num- re-views as well as in reviews of similar users [35]. In ad-
ber of nights, desired amenities, purpose of travel, etc.-the dition, the system could also extract and present, for each
output could be personalized not only in terms of the rec- recommended item, the experiences of other people who
ommended items, but also tailored specifically to support are interested in the same attributes as the current user.
the user’s needs. Stating a preference for “fitness center” Such a net-work of “co-staying in hotels” could thus intro-
could lead to information such as opening hours, available duce a novel way of increasing the interaction with RS.
machines, and pricing information being displayed more
prominently, or even further content being embedded, e.g. a Overall, as the issues mentioned before suggest, recom-
map with related workout options nearby. mendations often lack transparency, and are therefore con-
sidered less trustworthy or not meeting the user’s situa-
In general, a RS should be able to select the features most tional needs [26, 40]. Thus, we argue that also their pre-
important for adequately personalizing the presentation sentation should be adapted to better suit the current user,
ac-cording to the user’s interests and his or her situation. for example by presenting customized summaries of the
There-fore, the system might also leverage the wealth of recommended items as well as by identifying and selecting
information contained in user-generated data (i.e. reviews, those features for personalization that are most important to
comments, tags, or individual ratings for hotel and room him or her.
characteristics) to present more relevant details about the
recommended items. To illustrate this point, consider some- Conclusion
one who is interested in venues that offer good Wi-Fi con- In this paper, we have summarized our experiences in the
nectivity. When browsing the results, he or she might find re-search area of interactive recommending. To structure
it useful to read reviews that specifically mention aspects the different concerns and design options for interactive RS,
such as connection speed and signal strength or that give we presented a framework that allowed us to review the
an overall quality assessment. To facilitate comparison, this literature with respect to those aspects that bear potential
information could be presented in form of a graphical scale for integrating the systems with additional means for inter-
depicting the proportion of people who rated the internet action and may contribute to increase their transparency.
connection positively versus those who rated it negatively. For each aspect, we discussed influential existing develop-
Since people usually have more than one requirement, a ments in order to derive challenges for advancing the field
RS that can identify the most interesting attributes for the of interactive recommending towards further improving user
user could enhance recommendations with such personal- experience. In line with that, we also provided an outlook on
ized summaries, thereby increasing their trustworthiness. some directly related future work we have planned.
10
References and User-Adapted Interaction 22, 1-2 (2012), 125–
[1] Gediminas Adomavicius and Alexander Tuzhilin. 2015. 150.
Context-aware recommender systems. In Recom- [10] Nofar Dali Betzalel, Bracha Shapira, and Lior Rokach.
mender systems handbook. Springer, 191–226. 2015. Please, Not Now!: A Model for Timing Rec-
[2] Xavier Amatriain, Josep M Pujol, Nava Tintarev, and ommendations. In Proceedings of RecSys ’15. ACM,
Nuria Oliver. 2009. Rate it again: increasing recom- 297–300.
mendation accuracy by user re-rating. In Proceedings [11] Marco de Gemmis, Pasquale Lops, Cataldo Musto,
of the third ACM conference on Recommender sys- Fedelucio Narducci, and Giovanni Semeraro. 2015.
tems. ACM, 173–180. Semantics-aware content-based recommender sys-
[3] Linas Baltrunas, Bernd Ludwig, Stefan Peer, and tems. In Recommender Systems Handbook. Springer,
Francesco Ricci. 2011. Context-aware places of in- 119–159.
terest recommendations and explanations. In DEMRA [12] Tim Donkers, Benedikt Loepp, and Jürgen Ziegler.
’11, Vol. 740. 19–26. 2015. Merging Latent Factors and Tags to Increase
[4] Shlomo Berkovsky, Tsvi Kuflik, and Francesco Ricci. Interactive Control of Recommendations. In Poster
2008. Mediation of user models for enhanced person- Proceedings of RecSys âĂŹ15.
alization in recommender systems. User Modeling and [13] Tim Donkers, Benedikt Loepp, and Jürgen Ziegler.
User-Adapted Interaction 18, 3 (2008), 245–286. 2016. Tag-Enhanced Collaborative Filtering for In-
[5] Dirk Bollen, Bart P Knijnenburg, Martijn C Willemsen, creasing Transparency and Interactive Control. In Pro-
and Mark Graus. 2010. Understanding choice over- ceedings UMAP ’16. ACM, 169–173.
load in recommender systems. In Proceedings RecSys [14] Michael D Ekstrand, Daniel Kluver, F Maxwell Harper,
’10. ACM, 63–70. and Joseph A Konstan. 2015. Letting users choose
[6] Svetlin Bostandjiev, John O’Donovan, and Tobias recommender algorithms: An experimental study. In
Höllerer. 2012. Taste-Weights: a visual interactive hy- Proceedings RecSys ’15. ACM, 11–18.
brid recommender system. In Proceedings of RecSys [15] Mehdi Elahi, Mouzhi Ge, Francesco Ricci, Shlomo
’12. ACM, 35–42. Berkovsky, and Massimo David. 2015. Interaction
[7] Shuo Chang, F Maxwell Harper, and Loren Terveen. Design in a Mobile Food Recommender System. In
2015. Using groups of items for preference elicitation Proceedings of IntRS ’15. 49–52.
in recommender systems. In Proceedings CSCW ’15. [16] Emden Gansner, Yifan Hu, Stephen Kobourov, and
ACM, 1258–1269. Chris Volinsky. 2009. Putting recommendations on the
[8] Li Chen and Pearl Pu. 2012a. Critiquing-based recom- map: visualizing clusters and relations. In Proceedings
menders: survey and emerging trends. User Modeling RecSys ’09. ACM, 345–348.
and User-Adapted Interaction 22, 1-2 (2012), 125– [17] Brynjar Gretarsson, John O’Donovan, Svetlin Bo-
150. standjiev, Christopher Hall, and Tobias Höllerer. 2010.
[9] Li Chen and Pearl Pu. 2012b. Critiquing-based recom- Smallworlds: visualizing social recommendations. In
menders: survey and emerging trends. User Modeling Computer Graphics Forum, Vol. 29. Wiley Online Li-
brary, 833–842.
11
[18] Ido Guy. 2015. Social recommender systems. In (2012), 101–123.
Recommender Systems Handbook. Springer, 511– [27] Yehuda Koren, Robert Bell, Chris Volinsky, and others.
543. 2009. Matrix factorization techniques for recommender
[19] Negar Hariri, Bamshad Mobasher, and Robin Burke. systems. Computer 42, 8 (2009), 30–37.
2014. Context adaptation in interactive recommender [28] Carol C Kuhlthau. 1991. Inside the search process:
systems. In Proceedings RecSys ’14. ACM, 41–48. Information seeking from the user’s perspective. J.
[20] F Maxwell Harper, Funing Xu, Harmanpreet Kaur, Am. Soc. Inf. Sci. 42, 5 (1991), 361.
Kyle Condiff, Shuo Chang, and Loren Terveen. 2015. [29] Johannes Kunkel, Benedikt Loepp, and Jürgen Ziegler.
Putting users in control of their recommendations. In 2015. 3D-Visualisierung zur Eingabe von Präferen-
Proceedings RecSys ’15. ACM, 3–10. zen in Empfehlungssystemen [3D visualization to elicit
[21] Anthony Jameson, Martijn C Willemsen, Alexander preferences in recommender systems]. Proceedings of
Felfernig, Marco de Gemmis, Pasquale Lops, Giovanni M&C âĂŹ15 (2015).
Semeraro, and Li Chen. 2015. Human decision mak- [30] Benedikt Loepp, Katja Herrmanny, and Jürgen Ziegler.
ing and recommender systems. In Recommender 2015a. Blended recommending: Integrating interac-
Systems Handbook. Springer, 611–648. tive information filtering and algorithmic recommender
[22] Gawesh Jawaheer, Peter Weller, and Patty Kostkova. techniques. In Proceedings of CHI ’15. ACM, 975–
2014. Modeling user preferences in recommender 984.
systems: A classification framework for explicit and im- [31] Benedikt Loepp, Katja Herrmanny, and Jürgen Ziegler.
plicit user feedback. ACM Transactions on Interactive 2015b. Merging Interactive Information Filtering
Intelligent Systems 4, 2 (2014), 8. and Recommender Algorithms–Model and Concept
[23] Nicolas Jones, Armelle Brun, and Anne Boyer. 2011. Demonstrator. In icom, Vol. 14. 5–17.
Comparisons instead of ratings: Towards more stable [32] Benedikt Loepp, Tim Hussein, and Jüergen Ziegler.
preferences. In Proceedings of WIIAT ’11, Vol. 1. IEEE, 2014. Choice-based preference elicitation for collabo-
451–456. rative filtering recommender systems. In Proceedings
[24] Bart P Knijnenburg, Niels JM Reijmer, and Martijn C of CHI ’14. ACM, 3085–3094.
Willemsen. 2011. Each to his own: how different users [33] Tariq Mahmood, Francesco Ricci, and Adriano Ven-
call for different interaction methods in recommender turini. 2009. Improving recommendation effectiveness:
systems. In Proceedings RecSys ’11. ACM, 141–148. Adapting a dialogue strategy in online travel planning.
[25] Bart P Knijnenburg, Martijn C Willemsen, Zeno Gant- J. of IT & Tourism 11, 4 (2009), 285–302.
ner, Hakan Soncu, and Chris Newell. 2012. Explaining [34] Gary Marchionini. 1997. Information seeking in elec-
the user experience of recommender systems. User tronic environments. Number 9. Cambridge University
Modeling and User-Adapted Interaction 22, 4-5 (2012), Press.
441–504. [35] Julian McAuley and Jure Leskovec. 2013. Hidden
[26] Joseph A Konstan and John Riedl. 2012. Recom- factors and hidden topics: understanding rating di-
mender systems: from algorithms to user experience. mensions with review text. In Proceedings RecSys ’13.
User Modeling and User-Adapted Interaction 22, 1-2 ACM, 165–172.
12
[36] Theodora Nanou, George Lekakos, and Konstantinos Paolo Viappiani. 2011. Usability guidelines for product
Fouskas. 2010. The effects of recommendationsâĂŹ recommenders based on example critiquing research.
presentation on persuasion and satisfaction in a movie In Recommender Systems Handbook. Springer, 511–
recommender system. Multimedia systems 16, 4-5 545.
(2010), 219–230. [42] Francesco Ricci, Lior Rokach, and Bracha Shapira.
[37] Julia Neidhardt, Rainer Schuster, Leonhard Seyfang, 2015. Recommender systems: introduction and
and Hannes Werthner. 2014. Eliciting the users’ un- challenges. In Recommender Systems Handbook.
known preferences. In Proceedings RecSys ’14. ACM, Springer, 1–34.
309–312. [43] Shilad Sen, Jesse Vig, and John Riedl. 2009. Tagom-
[38] Bottyán Németh, Gábor Takács, István Pilászy, and menders: connecting users to items through tags. In
Domonkos Tikk. 2013. Visualization of movie features Proceedings of WWW ’09. ACM, 671–680.
in collaborative filtering. In Proceedings of SoMeT ’13. [44] Barry Smyth and Lorraine McGinty. 2003. An anal-
IEEE, 229–233. ysis of feedback strategies in conversational recom-
[39] Maria Augusta SN Nunes and Rong Hu. 2012. menders. In Proceedings of AICS ’03.
Personality-based recommender systems: an [45] Katrien Verbert, Denis Parra, Peter Brusilovsky, and
overview. In Proceedings RecSys ’12. ACM, 5–6. Erik Duval. 2013. Visualizing recommendations to
[40] Pearl Pu, Li Chen, and Rong Hu. 2012. Evaluating support exploration, transparency and controllability. In
recommender systems from the userâĂŹs perspective: Proceedings of IUI ’13. ACM, 351–362.
survey of the state of the art. User Modeling and User- [46] Jesse Vig, Shilad Sen, and John Riedl. 2011. Navigat-
Adapted Interaction 22, 4-5 (2012), 317–355. ing the tag genome. In Proceedings of IUI ’11. ACM,
[41] Pearl Pu, Boi Faltings, Li Chen, Jiyong Zhang, and 93–102.
13