<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Towards a Design Space for Personalizing the Presentation of Recommendations</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Catalin-Mihai Barbu</string-name>
          <email>catalin.barbu@uni-due.de</email>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Jurgen Ziegler</string-name>
          <email>juergen.ziegler@uni-due.de</email>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Introduction &amp;</institution>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>University of Duisburg-Essen Duisburg</institution>
          ,
          <country country="DE">Germany</country>
        </aff>
      </contrib-group>
      <fpage>10</fpage>
      <lpage>17</lpage>
      <abstract>
        <p>Although personalization plays a major role in the development of recommender systems, the presentation of recommendations{and especially the way in which it can be adapted to suit the user's needs{ has received relatively little attention from the research community. We introduce a design space for personalizing the presentation of recommendations and propose several dimensions that should be a part of it. Moreover, we present our initial insights about possible interactive mechanisms as well as potential evaluation criteria. Our goal is to provide a systematic way of designing personalized recommendation content, which should prove bene cial for other researchers working on this topic. In the longer term, we are interested to investigate whether such personalized presentation implementations in uence the perceived trustworthiness of the recommendations.</p>
      </abstract>
      <kwd-group>
        <kwd>Recommender systems</kwd>
        <kwd>personalization</kwd>
        <kwd>design space</kwd>
        <kwd>interactive control</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>Motivation</title>
      <p>
        Personalization is an important aspect of recommender systems (RS). It allows
websites and other Internet services to cater to individual tastes, interests, and
preferences. For many years, objective accuracy was considered one of the most
important criteria for ranking RS [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ]. Consequently, the use of personalization
was mostly focused on improving the algorithms and models used to generate
result sets. However, recommendations are only as good as users perceive them to
be. More recently, some researchers have begun to argue that subjective accuracy
is equally, if not more, important than objective accuracy and may play a larger
role in determining user satisfaction [
        <xref ref-type="bibr" rid="ref11 ref3">3, 11</xref>
        ]. Perceived accuracy has been shown
to be in uenced positively by user-related aspects such as control, trust, and
transparency [
        <xref ref-type="bibr" rid="ref13 ref3">3, 13</xref>
        ]. Personalization is already one of the methods used to help
users understand why a recommendation is suitable for them. Previous research
has investigated its positive in uence on user experience [
        <xref ref-type="bibr" rid="ref10 ref17">10, 17</xref>
        ]. Combining
personalization techniques with novel approaches from the eld of interactive
RS could therefore lead to additional insights into how user satisfaction can be
increased even further.
      </p>
      <p>A relatively unexplored topic in the eld of RS is the personalization of the
presentation of recommended items. Once user preferences have been elicited
(either implicitly or explicitly), this information can be used not only to
suggest personalized predictions, but also to customize the way in which they are
presented to the user. Adapting the presentation to t the user's needs has the
potential to open novel interaction possibilities for users and might provide useful
insights into the way in which people interact with RS. Against this background,
exploring the design space for the personalization of recommendations is a useful
research endeavor and an important step towards the implementation of a
prototype. The goal of this paper is to introduce a design space for personalizing the
presentation of recommendations and to present the dimensions that comprise
it.</p>
      <p>The remainder of the paper is structured as follows: We discuss related work
in Section 2, before proceeding to present the design space in Section 3. We
subsequently introduce some preliminary interactive mechanisms and evaluation
criteria. Finally, we discuss possible limitations and directions for future research
in Section 4.
2</p>
    </sec>
    <sec id="sec-2">
      <title>Related Work</title>
      <p>
        Personalization is well-studied in the eld of RS. Some of the main research foci
include deciding, for a given recommendation, what information to present, when
to present it, how much of it to present, and in what way. For instance, di
erent information modalities (such as various types of result lists or combinations
of text and images) have been compared to observe their e ect on the
persuasiveness of recommendations and on the users' satisfaction [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ]. Prior work has
also investigated models for context-aware RS that can predict the best time
to show recommendations [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ]. Other researchers have determined the number of
items in a result set that maximizes choice satisfaction without increasing choice
di culty [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ].
      </p>
      <p>
        Many existing approaches to personalizing the presentation of
recommendations rely on explanations [
        <xref ref-type="bibr" rid="ref13 ref16 ref19">13, 16, 19</xref>
        ]. \Common sense" approaches, which use
rules to determine what items to recommend and how to personalize the
presentation have also been developed [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]. Novel approaches for visualizing
recommendations have been proposed, such as those implemented in TasteWeights [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]
and TalkExplorer [
        <xref ref-type="bibr" rid="ref18">18</xref>
        ]. These interactive approaches a ord a certain degree of
control over the recommendation process to elicit feedback and preferences as
well as to increase transparency. The e ects of personalization, especially with
respect to the use of explanations, have been investigated in several prior works
(see, e.g., [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ] and [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ]).
      </p>
      <p>
        Previous research into design spaces for adaptive user interfaces highlighted
the importance of control over the adaptation algorithm and the importance
of adequate measures for user evaluation [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ]. This research focused on generic
user interface control structures (e.g., menus) and did not cover information
systems such as RS. To the best of our knowledge, no prior work has so far
focused explicitly on exploring the design space of personalized presentation of
recommendations.
3
      </p>
    </sec>
    <sec id="sec-3">
      <title>Analysis of Design Space</title>
      <p>We identify the following dimensions that comprise the design space (Fig. 1):
modality, salience, comparison functions, interactive control, explanations, and
trust cues. Each of these is explained in further detail below.</p>
      <p>The design space is meant to be applicable to numerous domains in which
RS are commonly used. To facilitate understanding of the various dimensions,
throughout this section we limit ourselves to using examples from the hotel
booking domain. Hotel recommendations are interesting for several reasons. First,
there is a higher risk associated with such choices{in comparison with movie
recommendations, for example. Risk arises, on the one hand, from the fact that
staying in a hotel typically costs a considerable amount of money. On the other
hand, there is also the risk associated with the e ects of a wrong recommendation
on the user's wellbeing. Second, the items in question have a reasonable set of
attributes that should be considered. These can be classi ed into hotel features
(e.g., location, price), room characteristics (e.g., bed size, number of electrical
sockets), and services (e.g., complimentary breakfast, free Wi-Fi). Third, there
is a large body of user-generated content, in the form of reviews, photos, tags,
and ratings, that can be leveraged in the presentation.
3.1</p>
      <sec id="sec-3-1">
        <title>Design Space Dimensions</title>
        <p>
          Modality refers to the form in which the content of the recommendation is
conveyed to the user. Information can be presented using text, graphical symbols,
audiovisual means, or combinations thereof [
          <xref ref-type="bibr" rid="ref4">4</xref>
          ]. Finding the most appropriate
modality for each type of content (for example, description, ratings, pricing
information, user reviews etc.) is an important aspect of personalization [
          <xref ref-type="bibr" rid="ref12">12</xref>
          ]. Some
users prefer to read an exhaustive description of the hotel to decide whether it
matches their requirements; others like viewing photos of the property.
Furthermore, some modalities may not be suitable for users with visual or auditory
impairments. Changing information modalities may also require that the system
adopt a di erent recommendation paradigm.
        </p>
        <p>
          Salience denotes the range of presentation features that are used to draw
users' attention. Particularly relevant information, such as attributes in which a
person is interested, should be emphasized. Conversely, less important features
might be shown in a subtler manner or even hidden altogether (e.g., business
services for vacationers). Standard presentation layouts, such as category-value
tables, can become di cult to parse if they exceed a certain number of rows.
Similarly, altering the size and color of text or using animations to highlight
important aspects can lead to information overload if used excessively. Instead,
a RS might re-order the list of attributes such that those that the user considers
most important are displayed at the top [
          <xref ref-type="bibr" rid="ref9">9</xref>
          ]. Relevant additional information can
also be shown directly. An example would be displaying the opening times of the
local gym to users who have expressed interest in tness (as opposed to simply
listing \ tness center" as a hotel amenity).
        </p>
        <p>
          Comparison functions help users evaluate item attributes and values across
di erent recommendations [
          <xref ref-type="bibr" rid="ref9">9</xref>
          ]. For example, consider people who enjoy spacious
accommodations. When browsing hotel recommendations, they would,
presumably, look speci cally for details about the size of the rooms. The same
information might be presented di erently by various vendors: as an area (e.g., \14
m2"); as the product of individual dimensions (e.g., \4x3.5 m:"); using di
erent units (e.g., \150 sq f t"); in relative terms (e.g., \standard size"); using a
blueprint on which the layout and dimensions of the room are depicted. In other
cases, such details might be missing altogether. A low comparability has a
detrimental e ect on the user's decision making processes as well as on her trust in
the generated results. The RS should therefore adapt the presentation such that
attribute values are normalized to facilitate comparison.
        </p>
        <p>
          Interactive control comprises the mechanisms through which a user in uences
the output of a RS. The complexity of the underlying algorithms that are used
to generate recommendations has increased a great deal over the years. For this
reason, many users associate modern RS with \black boxes" [
          <xref ref-type="bibr" rid="ref13">13</xref>
          ]. The lack of
transparency and limited options for controlling the output are frequently cited
as reasons for the users' lack of trust in the recommended items [
          <xref ref-type="bibr" rid="ref13">13</xref>
          ]. Various
approach for increasing user control have been proposed, ranging from novel
ways to elicit preferences [
          <xref ref-type="bibr" rid="ref7">7</xref>
          ] to innovative frameworks for enhancing decision
support [
          <xref ref-type="bibr" rid="ref9">9</xref>
          ]. A straightforward example would be a hotel RS that allows users to
modify the relative weights of hotel attributes per their own preferences.
        </p>
        <p>
          Explanations allow users to discover supporting evidence for a presented
attribute or claim and are one of the more common methods for increasing
transparency in RS [
          <xref ref-type="bibr" rid="ref14 ref16 ref19">14, 16, 19</xref>
          ]. A hotel description claiming that the establishment
is close to the city center may be misleading. It might measure only the
distance to the edge of the central district (rather than the geographical center),
or simply provide a \straight-line" distance that is of little help in practice. If
location is an important aspect for the user, the RS might, for example, display
the average walking time based on information extracted from user reviews or
from local transportation websites. More interactive approaches could leverage
GPS data to display a map that allows her to calculate the travel time between
the hotel and various landmarks, perhaps even using various means of
transportation. Providing su cient evidence is important for both objective as well
as subjective information. The former might be wrong or incomplete, whereas
the latter might need to be put into the proper context. Explanations could also
help clarify why a certain piece of information is presented{as well as why it is
presented in a certain way. To achieve this, the RS should be able to represent
the user's personalization pro le in a meaningful way.
        </p>
        <p>
          Trust cues are interface elements that allow the user to determine the
reliability of the presented information [
          <xref ref-type="bibr" rid="ref16">16</xref>
          ]. Item descriptions should be
complemented, to the extent possible, by objective measurements. The credibility of
user-generated content, such as user reviews, should also be evaluated. When
personalizing the presentation of a recommendation, a RS might show
supporting evidence contributed by trustworthy reviewers. This means ensuring, on the
one hand, that a review is not fake, and on the other hand, that the reviewer
has su cient expertise. It is, however, equally important to recognize that the
system has limited knowledge of its users' (personalization) preferences. Hence,
the RS should provide adequate trust cues to make the user aware of this
inherent uncertainty. In other words, a person who is considering a recommendation
should understand how trustworthy each part of the recommendation is. This
ensures that the user's perceived trustworthiness of the RS remains in sync with
the system's actual trustworthiness [
          <xref ref-type="bibr" rid="ref16">16</xref>
          ]. As an example, consider the case where
multiple reviewers have complained about the sti ness of the bed in a hotel that
otherwise appears to be a good match for a prospective traveler. The reliability
of this piece of information depends on how long ago the reviews were written,
on the proportion of guests who made similar comments, as well as on their
breadth of travel experience. All this should be considered and presented to the
user in a transparent manner.
3.2
        </p>
      </sec>
      <sec id="sec-3-2">
        <title>Interactive Mechanisms</title>
        <p>
          A promising approach for personalizing the presentation of recommendations is
to employ interactive mechanisms that support the user's decision making
process [
          <xref ref-type="bibr" rid="ref9">9</xref>
          ]. Instead of simply ordering reviews by date, a RS might preselect reviews
from people who have commented on issues that match the user's interests.
Initially, only the most relevant comments would be shown, though the user would
be a orded the option to expand each review fully. Going a step further, a RS
might o er \personalized summaries" containing relevant attributes, aggregate
ratings, review snippets, as well as relevant photos or maps.
        </p>
        <p>The content of recommendations is typically organized into sections, such
as general description, listing of attributes, ratings, tags, reviews or comments,
and photos. It is reasonable to expect that users' preferences extend also to the
order in which these sections are presented. For example, one might consider user
reviews more relevant than the owner-supplied description of the hotel. Hence,
the RS could allow users to customize the various content sections to their liking.
Furthermore, the system might also attempt to match users' expectations based
on available context information.</p>
        <p>Further interactive mechanisms could be developed to facilitate users' control
over their own personalization pro les. Ideally, the system should not only
provide the means for users to edit their pro les, but also to preview the e ect that
a prospective change would have on the presentation of the recommendations.
3.3</p>
      </sec>
      <sec id="sec-3-3">
        <title>Evaluation Criteria</title>
        <p>Based on the dimensions presented above, it seems possible to devise methods
for evaluating RS with respect to how strongly the di erent dimensions are
perceived by users. We believe that the main criterion for evaluating user interfaces
that implement the design space is their suitability with respect to the user's
informational need. This may depend on several factors, such as the consequences
of choosing wrongly (e.g., in terms of monetary costs or the user's wellbeing),
the required level of detail (i.e. how accurate does the information need to be),
and user characteristics.
4</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>Discussion and Future Work</title>
      <p>Personalizing the presentation of recommended items using interactive
mechanisms can lead to increased transparency and control over the recommendation
process. Since both aspects are central to the issue of trust, this additional kind
of personalization might increase the perceived trustworthiness of the
recommendations. However, this will need to be investigated empirically.</p>
      <p>A limiting factor for the design of such interactive and personalized
presentations is the quality of the user data, such as elicited preferences, that is
available to the RS. At the same time, many of the existing user models are not
optimized su ciently to support this level of customization. Therefore, one of
our planned directions for future research is to investigate how the user models
commonly used in RS can be expanded to a ord personalized presentations of
recommendations.</p>
      <p>Although we decided to focus in this paper only on the presentation of
individual recommendations, personalized presentations also make sense for the item
sets (i.e. even before the user has selected a recommendation for closer scrutiny).
We believe that the design space can be employed successfully in this situation,
though a more thorough examination of this case is required.</p>
      <p>Finally, we plan to validate the design dimensions presented in this paper
and to develop a prototype implementation of the design space on top of an
existing platform for hotel recommendations.</p>
    </sec>
    <sec id="sec-5">
      <title>Acknowledgments</title>
      <p>This work is supported by the German Research Foundation (DFG) under grant
No. GRK 2167, Research Training Group \User-Centred Social Media".</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <given-names>D.</given-names>
            <surname>Bollen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B. P.</given-names>
            <surname>Knijnenburg</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. C.</given-names>
            <surname>Willemsen</surname>
          </string-name>
          , and
          <string-name>
            <given-names>M.</given-names>
            <surname>Graus</surname>
          </string-name>
          .
          <article-title>Understanding choice overload in recommender systems</article-title>
          .
          <source>In Proceedings of the Fourth ACM Conference on Recommender Systems, RecSys '10</source>
          , pages
          <fpage>63</fpage>
          {
          <fpage>70</fpage>
          , New York, NY, USA,
          <year>2010</year>
          . ACM.
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <given-names>S.</given-names>
            <surname>Bostandjiev</surname>
          </string-name>
          ,
          <string-name>
            <surname>J. O'Donovan</surname>
            , and
            <given-names>T.</given-names>
          </string-name>
          <string-name>
            <surname>Ho</surname>
          </string-name>
          <article-title>llerer. Tasteweights: A visual interactive hybrid recommender system</article-title>
          .
          <source>In Proceedings of the Sixth ACM Conference on Recommender Systems, RecSys '12</source>
          , pages
          <fpage>35</fpage>
          {
          <fpage>42</fpage>
          , New York, NY, USA,
          <year>2012</year>
          . ACM.
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <given-names>Y.</given-names>
            <surname>Chen</surname>
          </string-name>
          .
          <article-title>Interface and interaction design for group and social recommender systems</article-title>
          .
          <source>In Proceedings of the Fifth ACM Conference on Recommender Systems, RecSys '11</source>
          , pages
          <fpage>363</fpage>
          {
          <fpage>366</fpage>
          , New York, NY, USA,
          <year>2011</year>
          . ACM.
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <given-names>P.</given-names>
            <surname>Cremonesi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Elahi</surname>
          </string-name>
          , and
          <string-name>
            <surname>F. Garzotto.</surname>
          </string-name>
          <article-title>User interface patterns in recommendation-empowered content intensive multimedia applications</article-title>
          .
          <source>Multimedia Tools Appl.</source>
          ,
          <volume>76</volume>
          (
          <issue>4</issue>
          ):
          <volume>5275</volume>
          {
          <fpage>5309</fpage>
          ,
          <string-name>
            <surname>Feb</surname>
          </string-name>
          .
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <given-names>N. Dali</given-names>
            <surname>Betzalel</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Shapira</surname>
          </string-name>
          , and
          <string-name>
            <given-names>L.</given-names>
            <surname>Rokach</surname>
          </string-name>
          .
          <article-title>"please, not now!": A model for timing recommendations</article-title>
          .
          <source>In Proceedings of the 9th ACM Conference on Recommender Systems, RecSys '15</source>
          , pages
          <fpage>297</fpage>
          {
          <fpage>300</fpage>
          , New York, NY, USA,
          <year>2015</year>
          . ACM.
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          <string-name>
            <surname>6. B. De Carolis</surname>
            , I. Mazzotta,
            <given-names>N.</given-names>
          </string-name>
          <string-name>
            <surname>Novielli</surname>
            , and
            <given-names>V.</given-names>
          </string-name>
          <string-name>
            <surname>Silvestri</surname>
          </string-name>
          .
          <article-title>Using common sense in providing personalized recommendations in the tourism domain</article-title>
          .
          <source>In In Proceedings of the Workshop on Context-Aware Recommender Systems</source>
          ,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <given-names>T.</given-names>
            <surname>Donkers</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Loepp</surname>
          </string-name>
          , and
          <string-name>
            <given-names>J.</given-names>
            <surname>Ziegler</surname>
          </string-name>
          .
          <article-title>Tag-enhanced collaborative ltering for increasing transparency and interactive control</article-title>
          .
          <source>In Proceedings of the 2016 Conference on User Modeling Adaptation and Personalization</source>
          ,
          <source>UMAP '16</source>
          , pages
          <fpage>169</fpage>
          {
          <fpage>173</fpage>
          , New York, NY, USA,
          <year>2016</year>
          . ACM.
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <given-names>L.</given-names>
            <surname>Findlater</surname>
          </string-name>
          and
          <string-name>
            <given-names>K. Z.</given-names>
            <surname>Gajos</surname>
          </string-name>
          .
          <article-title>Design space and evaluation challenges of adaptive graphical user interfaces</article-title>
          .
          <source>AI Magazine</source>
          ,
          <volume>30</volume>
          (
          <issue>4</issue>
          ):
          <volume>68</volume>
          {
          <fpage>73</fpage>
          ,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <given-names>A.</given-names>
            <surname>Jameson</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. C.</given-names>
            <surname>Willemsen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Felfernig</surname>
          </string-name>
          , M. de Gemmis, P. Lops, G. Semeraro, and
          <string-name>
            <given-names>L.</given-names>
            <surname>Chen</surname>
          </string-name>
          .
          <article-title>Human decision making and recommender systems</article-title>
          . In F. Ricci,
          <string-name>
            <given-names>L.</given-names>
            <surname>Rokach</surname>
          </string-name>
          , and B. Shapira, editors,
          <source>Recommender Systems Handbook</source>
          , pages
          <volume>611</volume>
          {
          <fpage>648</fpage>
          . Springer,
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <given-names>B. P.</given-names>
            <surname>Knijnenburg</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. C.</given-names>
            <surname>Willemsen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Z.</given-names>
            <surname>Gantner</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Soncu</surname>
          </string-name>
          , and
          <string-name>
            <given-names>C.</given-names>
            <surname>Newell</surname>
          </string-name>
          .
          <article-title>Explaining the user experience of recommender systems</article-title>
          .
          <source>User Modeling</source>
          and
          <string-name>
            <surname>User-Adapted</surname>
            <given-names>Interaction</given-names>
          </string-name>
          ,
          <volume>22</volume>
          (
          <issue>4-5</issue>
          ):
          <volume>441</volume>
          {
          <fpage>504</fpage>
          ,
          <string-name>
            <surname>Oct</surname>
          </string-name>
          .
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <given-names>J. A.</given-names>
            <surname>Konstan</surname>
          </string-name>
          and
          <string-name>
            <given-names>J.</given-names>
            <surname>Riedl</surname>
          </string-name>
          .
          <article-title>Recommender systems: From algorithms to user experience. User Modeling</article-title>
          and
          <string-name>
            <surname>User-Adapted</surname>
            <given-names>Interaction</given-names>
          </string-name>
          ,
          <volume>22</volume>
          (
          <issue>1-2</issue>
          ):
          <volume>101</volume>
          {
          <fpage>123</fpage>
          ,
          <string-name>
            <surname>Apr</surname>
          </string-name>
          .
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12. T. Nanou, G. Lekakos, and
          <string-name>
            <given-names>K.</given-names>
            <surname>Fouskas</surname>
          </string-name>
          .
          <article-title>The e ects of recommendations' presentation on persuasion and satisfaction in a movie recommender system</article-title>
          .
          <source>Multimedia Syst</source>
          .,
          <volume>16</volume>
          (
          <issue>4-5</issue>
          ):
          <volume>219</volume>
          {
          <fpage>230</fpage>
          ,
          <string-name>
            <surname>Aug</surname>
          </string-name>
          .
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13.
          <string-name>
            <surname>J. O'Donovan</surname>
            and
            <given-names>B.</given-names>
          </string-name>
          <string-name>
            <surname>Smyth</surname>
          </string-name>
          .
          <article-title>Trust in recommender systems</article-title>
          .
          <source>In Proceedings of the 10th International Conference on Intelligent User Interfaces</source>
          ,
          <source>IUI '05</source>
          , pages
          <fpage>167</fpage>
          {
          <fpage>174</fpage>
          , New York, NY, USA,
          <year>2005</year>
          . ACM.
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14.
          <string-name>
            <given-names>P.</given-names>
            <surname>Pu</surname>
          </string-name>
          and
          <string-name>
            <given-names>L.</given-names>
            <surname>Chen</surname>
          </string-name>
          .
          <article-title>Trust-inspiring explanation interfaces for recommender systems</article-title>
          .
          <source>Know.-Based Syst.</source>
          ,
          <volume>20</volume>
          (
          <issue>6</issue>
          ):
          <volume>542</volume>
          {
          <fpage>556</fpage>
          ,
          <string-name>
            <surname>Aug</surname>
          </string-name>
          .
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          15.
          <string-name>
            <given-names>P.</given-names>
            <surname>Pu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Chen</surname>
          </string-name>
          , and
          <string-name>
            <given-names>R.</given-names>
            <surname>Hu</surname>
          </string-name>
          .
          <article-title>Evaluating recommender systems from the user's perspective: Survey of the state of the art</article-title>
          .
          <source>User Modeling</source>
          and
          <string-name>
            <surname>User-Adapted</surname>
            <given-names>Interaction</given-names>
          </string-name>
          ,
          <volume>22</volume>
          (
          <issue>4-5</issue>
          ):
          <volume>317</volume>
          {
          <fpage>355</fpage>
          ,
          <string-name>
            <surname>Oct</surname>
          </string-name>
          .
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          16.
          <string-name>
            <given-names>D.</given-names>
            <surname>Sacha</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Senaratne</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B. C.</given-names>
            <surname>Kwon</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G. P.</given-names>
            <surname>Ellis</surname>
          </string-name>
          , and
          <string-name>
            <given-names>D. A.</given-names>
            <surname>Keim</surname>
          </string-name>
          .
          <article-title>The role of uncertainty, awareness, and trust in visual analytics</article-title>
          .
          <source>IEEE Trans. Vis. Comput. Graph.</source>
          ,
          <volume>22</volume>
          (
          <issue>1</issue>
          ):
          <volume>240</volume>
          {
          <fpage>249</fpage>
          ,
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          17.
          <string-name>
            <given-names>N.</given-names>
            <surname>Tintarev</surname>
          </string-name>
          and
          <string-name>
            <given-names>J.</given-names>
            <surname>Mastho</surname>
          </string-name>
          .
          <article-title>Designing and evaluating explanations for recommender systems</article-title>
          . In F. Ricci,
          <string-name>
            <given-names>L.</given-names>
            <surname>Rokach</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Shapira</surname>
          </string-name>
          , and P. B. Kantor, editors,
          <source>Recommender Systems Handbook</source>
          , pages
          <volume>479</volume>
          {
          <fpage>510</fpage>
          . Springer,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          18.
          <string-name>
            <given-names>K.</given-names>
            <surname>Verbert</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Parra</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Brusilovsky</surname>
          </string-name>
          , and
          <string-name>
            <given-names>E.</given-names>
            <surname>Duval</surname>
          </string-name>
          .
          <article-title>Visualizing recommendations to support exploration, transparency and controllability</article-title>
          .
          <source>In Proceedings of the 2013 International Conference on Intelligent User Interfaces</source>
          ,
          <source>IUI '13</source>
          , pages
          <fpage>351</fpage>
          {
          <fpage>362</fpage>
          , New York, NY, USA,
          <year>2013</year>
          . ACM.
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          19.
          <string-name>
            <surname>J. Vig</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          <string-name>
            <surname>Sen</surname>
            , and
            <given-names>J.</given-names>
          </string-name>
          <string-name>
            <surname>Riedl</surname>
          </string-name>
          . Tagsplanations:
          <article-title>Explaining recommendations using tags</article-title>
          .
          <source>In Proceedings of the 14th International Conference on Intelligent User Interfaces</source>
          ,
          <source>IUI '09</source>
          , pages
          <fpage>47</fpage>
          {
          <fpage>56</fpage>
          , New York, NY, USA,
          <year>2009</year>
          . ACM.
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>