<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>\I'd like it to do the opposite": Music-Making Between Recommendation and Obstruction</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Peter Knees</string-name>
          <email>peter.knees@jku.at</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Kristina Andersen</string-name>
          <email>kristina@steim.nl</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Marko Tkalcic</string-name>
          <email>marko.tkalcic@jku.at</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Dept. of Computational Perception, Johannes Kepler University Linz</institution>
          ,
          <country country="AT">Austria</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Studio for Electro-Instrumental Music (STEIM)</institution>
          ,
          <addr-line>Amsterdam</addr-line>
          ,
          <country country="NL">the Netherlands</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>To build new and rethink existing interfaces for producing and working creatively with electronic music, we are engaged in an ongoing conversation with professional musicians. With regard to getting suggestions on musical material and progression by the machine on, e.g., samples, loops, rhythmic patterns, or musical structure, we nd evidence that the notion of an algorithmic recommendation system should be extended through the use of artistic obstruction. We propose the concept of \strangeness" as an addition to recommendation systems to allow the adjustment of the degree of desired otherness in the suggestions made. This marks an important di erence to existing consumer-centred recommendation approaches, going even beyond the notion of serendipity.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>Motivation</title>
      <p>
        The majority of today's electronic music is created from pre-recorded or
livegenerated sound material. This process often combines sound loops and samples
with synthesized and processed elements. For supporting the creator in the
process of nding suited sound material to express musical ideas, content-based
audio recommendation and retrieval techniques can be applied, e.g., [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ].
However, these methods are often tailored to music consumers rather than creators,
resulting is limited uptake in the professional domain. To increase acceptance of
these methods also for the creative acts of music making and composing, within
the GiantSteps project,1 we are engaging in an ongoing conversation with
professional musicians to tailor musical tools to their needs.
      </p>
      <p>From 16 open-ended interviews with young electronic musicians and
producers conducted in the context of the Red Bull Music Academy 2014, a number
of ideas have emerged with regard to systems for collaboration, intelligent and
intuitive organisation of personal sound collections, and intelligent composition
support. In this paper we highlight one particular idea that emerged in the
context of discussing the usefulness of automatic suggestions on musical material
and progression by the machine, e.g., samples, loops, rhythmic patterns, or
musical structure, namely the desire to extend algorithmic recommendation systems
with the concept of artistic obstruction. We present exemplary quotes from the
1 http://giantsteps-project.eu
interviews as evidence and contextualise the ndings with respect to artistic
practise and recommender systems research. Finally, we propose the concept of
\strangeness" as an addition to recommendation systems to allow the
adjustment of the degree of desired otherness in the suggestions made. This request for
the creative domain marks an important di erence to existing consumer-centred
music recommendation approaches but might also inspire new developments in
mainstream recommendation systems.
2</p>
    </sec>
    <sec id="sec-2">
      <title>Recommendation for Creatives</title>
      <p>In relation to supportive and recommendation systems, i.e., to the question \how
do we want the computer to `help' us in our creative work process?", beside
issues of artistic control and the fear of making predictable sounds, it becomes
apparent that the desired features of recommenders in a creative context go
beyond the query-by-example-centred paradigm of nding similar items and even
also beyond the goal of serendipitous suggestions.</p>
      <p>\Part of making music is about being lost a little bit and accidentally
stumbling upon stu that you didn't think would work." USER07
This rst quote highlights a key element of much creative work, the
element of the accidental, sometimes caused by the positive and negative e ects
of malfunctioning of sound-editing software. It shows that serendipity is highly
important to support creative work.</p>
      <p>\It could be something like Google image search for example. [...] So if I
set it to 100% precise I want it to nd exactly what I am searching for
and probably I will not nd anything, but maybe if I instruct him for
15% and I input a beat or a musical phrase and it searches my samples
for that, that could be interesting." USER03</p>
      <p>Starting from an established retrieval scenario, the artist expresses the
preference that the di erence between the query and the results is an explicit factor
that can be controlled.</p>
      <p>\What I would probably rather want it would do is make it complex in a
way that I appreciate, like I would be more interested in something that
made me sound like the opposite of me ... but within the boundaries of
what I like, because that's useful. Cause I can't do that on my own, it's
like having a band mate basically. [...] I am happy for it to make
suggestions, especially if I can ignore them, but for it to just make suggestions
that I would have never come up with, but would wish that I had come
up with." USER07
\I'd like it to do the opposite actually, because the point is to get a
possibility, I mean I can already make it sound like me, it's easy." USER01</p>
      <p>These statements bear again the notion of controlled di erence, including
the idea of complete opposition. Instead of wanting the machine to propose a
common or well understood solution, the artists would like it to perform the role
of \the other", providing an alternative or divergent view on a piece of music. So
the desired functionality of the machine is to provide an alter-ego of sorts, which
provides the artist with opposite suggestions, that still reside within the artist's
idea of his own personal style. Note that this is in contrast to many
contentbased recommendation use cases and systems that try to mimic and predict the
behaviour of the user.</p>
      <p>\The listening from outside the room is such a good one, turning around
and the stereo just ipped, in terms of squinting, I often do this thing
of listening through, I work on a track for half an hour through like
horri c lters, so like the EQ feels all fucked up and putting it through
a gramophone, and then just switching it o ..." USER16</p>
      <p>Here a user lists a number of artistic strategies of \obstruction" he is already
using to assess the quality of a piece, by changing the perception of the freshly
edited music through changes in acoustics and hardware to render the piece
\strange".
3</p>
    </sec>
    <sec id="sec-3">
      <title>Contextualisation</title>
      <p>
        The idea of making things di erent and \strange" is a well-known cornerstone in
modern art strategies. The technique is known as \defamiliarisation" as de ned
by Shklovsky [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ] and is a basic artistic strategy central to both Surrealism and
Dada. It is based on the idea that the act of experiencing something occurs
inside the moment of perceiving it and that the further you confuse or otherwise
prolong the moment of arriving at an understanding, the deeper or more detailed
that understanding will be. This technique and the ndings from the interviews
can be directly translated into new requirements for recommendation engines in
music making.
      </p>
      <p>
        This need for opposition goes far beyond the commonly known and often
addressed needs for diversity, novelty, and serendipity in recommendation system
research, which has identi ed purely similarity-based recommendation as a
shortcoming that leads to decreased user satisfaction and monotony [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ]. This
phenomenon spans all domains: from news articles [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ] to photos [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ] to movies [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ] to
music recordings [
        <xref ref-type="bibr" rid="ref15 ref4">4,15</xref>
        ]. One idea proposed to increase diversity is to subvert the
basic idea of collaborative ltering systems of recommending what people with
similar interests found interesting by recommending the opposite of what the
least similar users (the k-furthest neighbors) want [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ]. Indeed it could be shown
that this technique allows to increase diversity among relevant suggestions.
      </p>
      <p>
        The common theme of this research is that diversity should not signi cantly
harm the accuracy of the approaches, i.e., items need to be relevant [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ],
however heterogeneous. Diversity is thus merely aimed at providing \the user with
optimal coverage of the information space in the vicinity of their query" [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ].
Ultimately, this still bears the notion of similarity to preserve the close context,
which might constitute one of the biggest di erences between media
consumption and creation, where challenge, change of context, and questioning are the
desired qualities rather than \more of the same".
      </p>
      <p>However, the aspect of recommending the opposite has not received much
attention, potentially caused by a lack or insu ciency of technical de nitions
of terms such as \unexpectedness", \serendipity", or \utility" (which is often
de ned through economic factors rather than a real value for the user | not to
speak of creative output) as well as the non-trivial notion of \the other", which
rst requires de ning the full space of possible actions. While proximity of data
items can be de ned rather intuitively based on a similarity metric (e.g., the
nearest neighbours), remoteness is a less applicable concept. Particularly in
highdimensional data spaces, similarity-based rankings become somewhat
meaningless at higher ranks, as many points can have comparable distances (however, in
di erent directions, making them mutually dissimilar, thus inconsistent).</p>
      <p>
        In the context of experimental music creation, Collins has addressed the
question of opposition in the \Contrary Motion" system [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ] using a low-dimensional
representation of rhythm. The system opposes a piano player's rhythm in real
time by constructing a structure located in the space of actions \where the
human is likely not to be" [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ]. The hypothesis underlying the system is that being
confronted with an oppositional music style can be stimulating for a musician.
Experiments where the opposing structure is soni ed using a di erent instrument
have indeed shown that musicians start to experiment and play with the
opposing agent. The quotes presented in section 2 further support this hypothesis.
4
      </p>
    </sec>
    <sec id="sec-4">
      <title>Conclusions and Proposed Concept</title>
      <p>Inspired by both the input from our expert users and the inherent di culty
in creating e ective search algorithms, and picking up the idea of controlled
di erence to the point of complete otherness in the context of electronic music
production, we propose to include \strangeness" as a controllable parameter,
e.g., through a dedicated dial or slider.</p>
      <p>
        The concept of \strangeness" is related to concepts like serendipity and
diversity [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ] and even more to unexpectedness [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ] in consumer-oriented recommender
systems in that the item space and its niches need to be explored in order to nd
desired results. However, while end consumers of, e.g., movies, might be more or
less inclined to watch something di erent and consider it a satisfactory result,
music composers and other creatives want to be challenged. For end users the
consumption ends with the item (i.e., at the end of a movie or a song) while
for artists the recommendation is just the beginning of a creative process that
will be assessed way after the recommendation has been given. This includes the
possibility of having a rst negative opinion about the recommendation (equal
to a low rating in consumer-based recommenders) in order to spark a creative
streak that eventually results in being content with the given recommendation
(high rating). Thus, for strangeness, the goal is not to optimize for immediate
liking or usefulness but primarily in giving results that help in re ecting upon
the whole process. Unexpectedness, as de ned in [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ], plays a role in this process,
however the ideas of controlled strangeness and desired opposition also
encompass the possibility of including \irrelevant" results, i.e., results whose \utility"
value might only become apparent after some unde ned time. \Strange results"
| as well as complete opposition, as the extreme form of strangeness | could
therefore also help in appreciating other results in a di erent way, after being
exposed to the strange version. This could be understood as a set recommendation
problem, where the user's utility function is uncertain, because of the vagueness
of the feedback provided eventually [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ].
      </p>
      <p>
        In terms of user interaction, this concept could, for instance, extend a
recommender system such that it allows the artist to query the machine based on
existing musical material, but instead of returning just similar sound les, it
facilitates setting the level of \strangeness" the artist would like for the results.
Using the same or a comparable algorithm as one would have traditionally used
for a recommendation of a similar le, the system hence lets the artist set an
explicit level of otherness, allowing to preserve artistic control between variation
and opposition. By re-introducing the art strategy of defamiliarisation we hope
to propose an interface for working creatively with music without creating a
system of sameness and consensus. We are aware that this leads to new questions
regarding (semantic) descriptions of similarity spaces and personalised and
contextual de nitions of \the other", which need further investigation to become
applicable. Another consideration that needs to be made, when factoring in the
ndings of this paper, is that the degree of opposition and strangeness desired
depends on the preferences and the working style of the music maker. Taking
personality traits into account is a growing topic in recommendation [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ] and
the proposed concept, being highly user-speci c in a domain where user
individuality is a key factor, would bene t from being de ned around user properties
and personality aspects from the very beginning.
      </p>
      <p>Beyond our use case, we also see controlled \strangeness" as a potential
extension for other, non-creative domains. As commercial recommenders have
become a ubiquitous (and seldom inspirational) commodity, opposition to the
user might be a way to make them exciting again and provoke interaction.
5</p>
    </sec>
    <sec id="sec-5">
      <title>Acknowledgments</title>
      <p>This work is supported by the European Union 7th Framework Programme
FP7/2007-2013 through the GiantSteps project (grant agreement no. 610591).</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <surname>Adamopoulos</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tuzhilin</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          :
          <article-title>On unexpectedness in recommender systems: Or how to better expect the unexpected</article-title>
          .
          <source>ACM Transactions on Intelligent Systems and Technology</source>
          <volume>5</volume>
          (
          <issue>4</issue>
          ),
          <volume>54</volume>
          :1{
          <fpage>54</fpage>
          :32 (Dec
          <year>2014</year>
          ), http://doi.acm.
          <source>org/10.1145/2559952</source>
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <surname>Casey</surname>
            ,
            <given-names>M.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Veltkamp</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Goto</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Leman</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rhodes</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Slaney</surname>
            ,
            <given-names>M.:</given-names>
          </string-name>
          <article-title>ContentBased Music Information Retrieval: Current Directions and Future Challenges</article-title>
          .
          <source>Proceedings of the IEEE 96, 668{696 (April</source>
          <year>2008</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <surname>Castells</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hurley</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Vargas</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          :
          <article-title>Novelty and diversity in recommender systems</article-title>
          . In: Ricci,
          <string-name>
            <given-names>F.</given-names>
            ,
            <surname>Rokach</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            ,
            <surname>Shapira</surname>
          </string-name>
          ,
          <string-name>
            <surname>B</surname>
          </string-name>
          . (eds.) Recommender Systems Handbook. Springer, 2nd edn. (
          <year>2015</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <surname>Celma</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Herrera</surname>
            ,
            <given-names>P.:</given-names>
          </string-name>
          <article-title>A new approach to evaluating novel recommendations</article-title>
          .
          <source>In: Proceedings of the 2008 ACM Conference on Recommender Systems</source>
          . pp.
          <volume>179</volume>
          {
          <fpage>186</fpage>
          . RecSys '08,
          <string-name>
            <surname>ACM</surname>
          </string-name>
          , New York, NY, USA (
          <year>2008</year>
          ), http://doi.acm.
          <source>org/10</source>
          .1145/ 1454008.1454038
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5. Collins, N.:
          <article-title>Contrary motion : An oppositional interactive music system</article-title>
          . In: Beilharz,
          <string-name>
            <given-names>K.</given-names>
            ,
            <surname>Bongers</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            ,
            <surname>Johnston</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            ,
            <surname>Ferguson</surname>
          </string-name>
          , S. (eds.)
          <source>Proceedings of the International Conference on New Interfaces for Musical Expression</source>
          . pp.
          <volume>125</volume>
          {
          <fpage>129</fpage>
          .
          <string-name>
            <surname>Sydney</surname>
          </string-name>
          ,
          <string-name>
            <surname>Australia</surname>
          </string-name>
          (
          <year>2010</year>
          ), http://www.nime.org/proceedings/2010/nime2010_
          <fpage>125</fpage>
          .pdf
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <given-names>Di</given-names>
            <surname>Noia</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            ,
            <surname>Ostuni</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.C.</given-names>
            ,
            <surname>Rosati</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            ,
            <surname>Tomeo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            ,
            <surname>Di Sciascio</surname>
          </string-name>
          ,
          <string-name>
            <surname>E.</surname>
          </string-name>
          :
          <article-title>An analysis of users' propensity toward diversity in recommendations</article-title>
          .
          <source>In: Proceedings of the 8th ACM Conference on Recommender Systems</source>
          . pp.
          <volume>285</volume>
          {
          <fpage>288</fpage>
          . RecSys '14,
          <string-name>
            <surname>ACM</surname>
          </string-name>
          , New York, NY, USA (
          <year>2014</year>
          ), http://doi.acm.
          <source>org/10</source>
          .1145/2645710.2645774
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <surname>Li</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Wang</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Li</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Knox</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Padmanabhan</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          :
          <article-title>Scene: A scalable twostage personalized news recommendation system</article-title>
          .
          <source>In: Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval</source>
          . pp.
          <volume>125</volume>
          {
          <fpage>134</fpage>
          . SIGIR '11,
          <string-name>
            <surname>ACM</surname>
          </string-name>
          , New York, NY, USA (
          <year>2011</year>
          ), http://doi.acm.
          <source>org/10</source>
          .1145/2009916.2009937
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <surname>Radu</surname>
            ,
            <given-names>A.L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ionescu</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Menendez</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          , Stottinger, J.,
          <string-name>
            <surname>Giunchiglia</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>De Angeli</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          :
          <article-title>A hybrid machine-crowd approach to photo retrieval result diversi cation</article-title>
          . In: Gurrin,
          <string-name>
            <given-names>C.</given-names>
            ,
            <surname>Hopfgartner</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            ,
            <surname>Hurst</surname>
          </string-name>
          ,
          <string-name>
            <given-names>W.</given-names>
            ,
            <surname>Johansen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            ,
            <surname>Lee</surname>
          </string-name>
          ,
          <string-name>
            <surname>H.</surname>
          </string-name>
          ,
          <string-name>
            <given-names>O</given-names>
            <surname>'Connor</surname>
          </string-name>
          , N. (eds.)
          <source>MultiMedia Modeling, Lecture Notes in Computer Science</source>
          , vol.
          <volume>8325</volume>
          , pp.
          <volume>25</volume>
          {
          <fpage>36</fpage>
          . Springer International Publishing (
          <year>2014</year>
          ), http://dx.doi.org/10.1007/ 978-3-
          <fpage>319</fpage>
          -04114-
          <issue>8</issue>
          _
          <fpage>3</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <surname>Said</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Fields</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Jain</surname>
            ,
            <given-names>B.J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Albayrak</surname>
            ,
            <given-names>S.:</given-names>
          </string-name>
          <article-title>User-centric evaluation of a k-furthest neighbor collaborative ltering recommender algorithm</article-title>
          .
          <source>In: Proceedings of the 2013 Conference on Computer Supported Cooperative Work</source>
          . pp.
          <volume>1399</volume>
          {
          <fpage>1408</fpage>
          . CSCW '13,
          <string-name>
            <surname>ACM</surname>
          </string-name>
          , New York, NY, USA (
          <year>2013</year>
          ), http://doi.acm.
          <source>org/10</source>
          .1145/2441776. 2441933
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <surname>Shklovsky</surname>
          </string-name>
          , V.B.:
          <article-title>Art as technique (</article-title>
          <year>1917</year>
          ). In: Lemon,
          <string-name>
            <given-names>L.T.</given-names>
            ,
            <surname>Reis</surname>
          </string-name>
          ,
          <string-name>
            <surname>M.J</surname>
          </string-name>
          . (eds.) Russian Formalist Criticism: Four Essays. University of Nebraska Press, Lincoln, USA (
          <year>1965</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <surname>Smyth</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>McClave</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          :
          <article-title>Similarity vs. diversity</article-title>
          .
          <source>In: Proceedings of the 4th International Conference on Case-Based Reasoning: Case-Based Reasoning Research and Development</source>
          . pp.
          <volume>347</volume>
          {
          <fpage>361</fpage>
          . ICCBR '
          <volume>01</volume>
          , Springer-Verlag, London, UK, UK (
          <year>2001</year>
          ), http://dl.acm.org/citation.cfm?id=
          <volume>646268</volume>
          .
          <fpage>758890</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <surname>Vargas</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Castells</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          :
          <article-title>Rank and relevance in novelty and diversity metrics for recommender systems</article-title>
          .
          <source>In: Proceedings of the Fifth ACM Conference on Recommender Systems</source>
          . pp.
          <volume>109</volume>
          {
          <fpage>116</fpage>
          . RecSys '11,
          <string-name>
            <surname>ACM</surname>
          </string-name>
          , New York, NY, USA (
          <year>2011</year>
          ), http://doi.acm.
          <source>org/10</source>
          .1145/2043932.2043955
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13.
          <string-name>
            <surname>Viappiani</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Boutilier</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          :
          <article-title>Regret-based optimal recommendation sets in conversational recommender systems</article-title>
          .
          <source>In: Proceedings of the Third ACM Conference on Recommender Systems</source>
          . pp.
          <volume>101</volume>
          {
          <fpage>108</fpage>
          . RecSys '09,
          <string-name>
            <surname>ACM</surname>
          </string-name>
          , New York, NY, USA (
          <year>2009</year>
          ), http://doi.acm.
          <source>org/10</source>
          .1145/1639714.1639732
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14.
          <string-name>
            <surname>Wu</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chen</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>He</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          :
          <article-title>Using personality to adjust diversity in recommender systems</article-title>
          .
          <source>In: Proceedings of the 24th ACM Conference on Hypertext and Social Media</source>
          . pp.
          <volume>225</volume>
          {
          <fpage>229</fpage>
          . HT '13,
          <string-name>
            <surname>ACM</surname>
          </string-name>
          , New York, NY, USA (
          <year>2013</year>
          ), http://doi.acm.
          <source>org/10</source>
          .1145/2481492.2481521
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          15.
          <string-name>
            <surname>Zhang</surname>
            ,
            <given-names>Y.C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Seaghdha</surname>
            ,
            <given-names>D.O.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Quercia</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Jambor</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          :
          <article-title>Auralist: Introducing serendipity into music recommendation</article-title>
          .
          <source>In: Proceedings of the Fifth ACM International Conference on Web Search and Data Mining</source>
          . pp.
          <volume>13</volume>
          {
          <fpage>22</fpage>
          . WSDM '12,
          <string-name>
            <surname>ACM</surname>
          </string-name>
          , New York, NY, USA (
          <year>2012</year>
          ), http://doi.acm.
          <source>org/10</source>
          .1145/2124295.2124300
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>