<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Accounting for Bossy Users in Context-Aware Group Recommendations</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>(Discussion Paper)</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Davide Azzalini</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Elisa Quintarelli</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Emanuele Rabosio</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Letizia Tanca</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Politecnico di Milano</institution>
          ,
          <addr-line>Milan</addr-line>
          ,
          <country country="IT">Italy</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>University of Verona</institution>
          ,
          <addr-line>Verona</addr-line>
          ,
          <country country="IT">Italy</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>Lots of activities, like watching a movie or going to the restaurant, are intrinsically group-based. To recommend such activities to groups, traditional single-user recommendation techniques are not appropriate and, as a consequence, over the years a number of group recommender systems have been developed. Recommending items to be enjoyed together by a group of people poses many ethical challenges: in fact, a system whose unique objective is to achieve the best recommendation accuracy might learn to disadvantage submissive users in favor of more aggressive ones. In this work we investigate the ethical challenges of context-aware group recommendations, in the general case of ephemeral groups (i.e., groups where the members might be together for the first time), using a method that can recommend also items that are new to the system. We show the goodness of our method on two real-world datasets. The first one is a very large dataset containing the personal and group choices regarding TV programs of 7,921 users w.r.t. sixteen contexts of viewing, while the second one gathers the musical preferences (both individual and in groups) of 280 real users w.r.t. two contexts of listening. Our extensive experiments show that our method always manages to obtain the highest recall while delivering ethical guarantees in line with the other fair group recommender systems tested.</p>
      </abstract>
      <kwd-group>
        <kwd>eol&gt;group recommender systems</kwd>
        <kwd>context-aware recommender systems</kwd>
        <kwd>computer ethics</kwd>
        <kwd>fairness</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>
        Recommender Systems are software tools and techniques that provide suggestions for items to
be of use to a user [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. Several everyday activities are intrinsically group-based, thus recent
research concentrates also on systems that suggest activities that can be performed together
with other people and are typically social. The group recommendation problem introduces
further challenges with respect to the traditional single-user recommendations: (i) the group
members may have diferent preferences, and finding items that meet the tastes of everyone may
be impossible; (ii) a group may be formed by people who happen to be together for the first time,
and, in this case, not being any history of the group’s preferences available, the recommendation
can only be computed on the basis of those known for the group members combined by means of
some aggregation function; (iii) last but not least, people, when in a group, may exhibit diferent
behaviors with respect to when they are alone, and therefore their individual preferences
sometimes might not be a reliable source of information. This last observation introduces an
unfairness problem: if the recommender system learns to consider the preferences of some
users as more relevant than those of the others, the overall satisfaction of the users belonging
to a group may not be optimal. This unbalance in the negotiation power that the system learns
to assign to diferent users, with the purpose of obtaining the best possible recommendation
accuracy, may be the result of unfair dynamics, e.g. some users being more aggressive and
some others not feeling confident enough to stand up for themselves. In this work we extend
a state-of-the-art system for context-aware recommendations to ephemeral groups based on
the concept of contextual influence [
        <xref ref-type="bibr" rid="ref2 ref3">2, 3</xref>
        ] to account also for fairness. Experiments on two
real-world datasets show that our approach outperforms seven other fair group recommender
systems by achieving a consistently better recall while providing similar ethical guarantees.
      </p>
    </sec>
    <sec id="sec-2">
      <title>2. Related Work</title>
      <sec id="sec-2-1">
        <title>Context-aware Recommender Systems</title>
        <p>
          The majority of the existing approaches to Recommender Systems do not take into
consideration any contextual information, however, in many applications, the context of use might be
fundamental in guiding the current preference of a user [
          <xref ref-type="bibr" rid="ref4">4</xref>
          ]. Recent studies have shown that
Context-Aware Recommender Systems can generate a very high increase in performance [
          <xref ref-type="bibr" rid="ref5">5</xref>
          ].
        </p>
      </sec>
      <sec id="sec-2-2">
        <title>Group Recommender Systems</title>
        <p>
          Group Recommender Systems are systems that produce a common recommendation for a group
of users [
          <xref ref-type="bibr" rid="ref6">6</xref>
          ]. Group recommendations works usually address two kinds of groups: persistent
and ephemeral [
          <xref ref-type="bibr" rid="ref7">7</xref>
          ]. Persistent groups contain people that have a previous significant history of
activities together, while ephemeral groups are formed by people who happen to be together for
the first time. In the case of persistent groups, classical recommendation techniques can be used,
since the group can be considered as a single user, whereas, in the case of ephemeral groups,
recommendations must be computed on the basis of those known for the members of the group.
A number of diferent aggregation strategies for the individual preferences have been proposed
over the years [
          <xref ref-type="bibr" rid="ref6">6</xref>
          ], however most of these aggregation strategies clearly violate the fairness
principles. For instance, maximum satisfaction, used in [8, 9, 10, 11, 12], chooses the item for
which the individual preference score is the highest, efectively ignoring the satisfaction of most
of the users in the group. Other clear examples of unfair aggregation strategies are works such
as [13, 14, 15], which assign a diferent power to group members based on their expertise.
        </p>
      </sec>
      <sec id="sec-2-3">
        <title>Fairness in Recommender Systems</title>
        <p>In single-user Recommender Systems, fairness is usually assessed with regard to sensitive
attributes which are generally prone to discrimination (e.g., gender, ethnicity or social belonging)
by verifying the presence of a discriminated class within the user set [16, 17]. When fairness
is evaluated considering Group Recommender Systems, it should be computed within groups.
Since the groups we consider in this work are composed of few users, evaluating fairness in the
way just described is not a suitable solution. Instead of detecting unfairness towards a protected
group of users, we aim to detect and prevent unfairness towards single users within a group
whose desires are not considered when forming a recommendation for the whole group.</p>
      </sec>
      <sec id="sec-2-4">
        <title>Fairness in Group Recommender Systems</title>
        <p>
          Some aggregation strategies exist that, despite not having been developed to explicitly address
ethical issues, aggregate individual preferences in a way that resembles fairness. Least misery,
used in [
          <xref ref-type="bibr" rid="ref7">7, 8, 18, 19, 9, 20, 10, 11, 12, 21</xref>
          ], chooses the items for which the lowest value among
the preferences of the group members is the greatest one. The authors in [22] introduce an
aggregation function which maximizes the satisfaction of group components, while, at the same
time, minimizes the disagreement among them. Average, used in [8, 23, 18, 19, 9, 10, 13, 11, 12, 21],
computes the group preference for an item as the arithmetic mean of the individual scores. Lastly,
some recent works try to explicitly target the aim of producing fair group recommendations. In
[24] the preferences of individual users are combined with a measure of fairness, to guarantee
that all the users are somehow satisfied. In [ 25, 26] two aggregation strategies are proposed: one
is based on the idea of proportionality, while the other one is based on the idea envy-freeness.
In [27] a greedy algorithm to achieve rank-sensitive balance is presented.
        </p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>3. The proposed method</title>
      <p>
        In this section we review a previous approach of ours, introduced in [
        <xref ref-type="bibr" rid="ref2 ref3">2, 3</xref>
        ], CtxInfl . Then, our
contribution to make CtxInfl more fair will be presented. The resulting method is named FARGO.
      </p>
      <sec id="sec-3-1">
        <title>3.1. CtxInfl</title>
        <p>We considere a set of items  and a set of users  , from which any group  ∈ ℘( ) can
be extracted.  is the set of possible contexts in the given scenario, where a context  is the
conjunction of a set of dimension/value pairs: e.g., for the TV dataset, a context might be
 = ⟨_ =  ∧  = ⟩. We assume the availability of a log ℒ
recording the history of the items previously chosen by groups formed in the past, where each
element of ℒ is a 4-ple ( ,  ,  ,  ),  being the time instant in which the item  ∈  has
been chosen by the group  ∈ ℘( ) in the context  ∈ . A contextual scoring function
(, , ), with  ∈  ,  ∈ ,  ∈ , assigning to each user the score given to the items in
the various contexts, is computed ofline on the basis of the log of the past individual choices
and of the item descriptions in terms of their attributes, using any context-aware recommender
system for single users from the literature.  (, , ) is the function that returns the list
of the  items preferred by user  in context , according to the values of (, , ), for
each  ∈  available at instant . Given a target group  ∈ ℘( ), a context  ∈  and a time
instant , the group recommendation is obtained by recommending to the users in  a list (i.e.,
an ordered set) of  items, considered interesting in context , from those items in  that are
available at time instant , according to the following procedure:</p>
        <sec id="sec-3-1-1">
          <title>3.1.1. Influence computation</title>
          <p>The group preference for an item is obtained by aggregating the individual preferences of the
group members on the basis of their influence. In each context , the influence  (, ) of a
given user  is derived ofline by comparing the behavior of  when alone (i.e., ’s individual
preferences) with ’s behaviors in groups (i.e., the interactions contained in the log ℒ). Basically,
the influence of  tells us how many times the groups containing  have selected one of ’s
favorite items. Let  (, , ) be the list of the  items preferred by user  in context ,
according to the values of (, , ) for each  ∈  available at instant . The contextual
influence is defined as follows:
 (, ) = | ∈ ℒ :  =  ∧  ∈  ∧  ∈  (, ,  )|</p>
          <p>| ∈ ℒ :  =  ∧  ∈  |
The value of  (, ) quantifies the ability of user  to direct the group’s decision towards
’s own tastes while in context .</p>
        </sec>
        <sec id="sec-3-1-2">
          <title>3.1.2. Top-K Group Recommendation Computation</title>
          <p>
            Top- recommendations are computed online, when a group of users requires that the system
suggests some interesting items to be enjoyed together. The system must compute the group
preferences for the items, and then determine the  items with the highest scores. Given a
group  ∈ ℘( ), its preference (, , ) for  ∈  in the context  ∈  is computed as
the average of the preferences of its members weighed on the basis of each member’s influence
(Eq. 1) in context :
(, , ) =
∑︀∈  (, ) · (, , )
∑︀∈  (, )
(1)
(2)
Then, the top- list of items preferred by a certain group  in context  at time instant  is
determined by retrieving the  items with the highest scores among those available at time .
3.2. FARGO
Being CtxInfl based on the concept of influence, it inevitably privileges the preferences of the
most influential users. As a consequence, the results of the recommendation process are biased
towards the preference of one user or few users of the group who can be considered as the
leaders, or, using a more contemporary word, “influencers". Our aim is to add an element of
fairness to CtxInfl while maintaining its general structure, which already proved to be very
eficient and scalable [
            <xref ref-type="bibr" rid="ref3">3</xref>
            ]. Among the various phases of CtxInfl on which we could act (i.e.,
individual preferences computation, influence computation, and Top-K group recommendations
computation), the last is the most suitable one, as it is the only one acting on groups. Following
this intuition, we propose to add a fairness factor to the computation of the score for each item
(Eq. 2), in order to modify the order of the items in the Top- list produced in such a way that
items representing unfair recommendations will not appear on top. This is further motivated by
the fact that, when people make decisions in groups, not necessarily they follow the decision
of a leader (as assumed by CtxInfl ): in some cases people may take decisions trying to satisfy
every group member as much as possible. This means that considering only influence may not
be a complete strategy even if we put aside our ethical concerns. In order not to increase the
complexity of the computation of Eq. 2, we build our fairness element using just the individual
contextual scores, which are already used to compute Eq. 2. We call consensus the metric that
quantifies how much the individual preferences of group members agree on the evaluation of
an item. The consensus of a group  on an item  in a context  is therefore defined as:
consensus(, , ) = 1 −
||
where (, , ) is the average score for item  among ’s members in context . The
consensus for an item for which users gave a similar evaluation will be close to 1, while it will
reach its minimum when very discordant scores are considered. According to the formula of
the maximum variance,  ∈ [0.75, 1]. After having defined , we propose
to integrate it in Eq. 2 in the following way:
,
(3)
∑︀∈ ︀( (, , ) − (, , ))︀ 2
 _(, , ) =
∑︀∈  (, ) · (, , )
∑︀∈  (, )
· consensus(, , )||
(4)
We exponentiate consensus to the group size (with the efect of further reducing the overall
score) according to the intuition that the magnitude of unfairness in group recommendations
is proportional to the group size. In fact, the bigger the group, the bigger the potential harm
produced by taking into consideration solely the leader/influencer’s will is.
          </p>
        </sec>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>4. Experimental Results</title>
      <p>In this section we present the results obtained by applying the proposed approach to two
diferent real-world datasets. To evaluate the recommendation performance we use recall,
considering for  (number of items to be recommended) the values 1, 2 and 3. To evaluate the
ethical properties of our method we used the two metrics proposed in [28] for estimating user
discrimination, called score disparity and recommendation disparity, adapted to our needs. Score
disparity is computed as the Gini coeficient of user satisfaction, i.e., the relative gain achieved
by the user due to the actual recommendation with respect to the optimal recommendation
strategy from the user perspective. Recommendation disparity is computed as the Gini coeficient
of user gains, i.e., how many of the recommended items match the user Top-K items.</p>
      <p>
        We compare our approach to the following methods: average (AVG) [8, 23, 18, 19, 9, 10, 13,
11, 12, 21], Fair Lin [24], Fair Prop [25, 26], Envy Free [25, 26], minimum disagreement (Dis)
[22], least misery (LM) [
        <xref ref-type="bibr" rid="ref7">7, 8, 18, 19, 9, 20, 10, 11, 12, 21</xref>
        ] and GFAR [27].
      </p>
      <sec id="sec-4-1">
        <title>4.1. TV Dataset</title>
        <p>This dataset contains TV viewing information related to 7,921 users and 119 channels,
broadcasted both over the air and by satellite. The dataset is composed of an Electronic Program
Guide (EPG) containing the description of 21,194 distinct programs, and a log containing both
individual and group viewings performed by the users. The log spans from December 2012 to</p>
        <p>Recall
37.94%
33.914%
33.22%
32.99%
29.33%
33.57%
30.35%
30.47%
February 2013 and contains 4,968,231 entries, among which we retained just the syntonizations
longer than three minutes. 3,519,167 viewings were performed by individual users, and are
used to compute the individual preferences of the group members. The remaining 1,449,064
viewings have been done by more than one person. The two context dimensions considered are
day of the week (weekday vs. weekend) and the time slot. The available values for the time slot
are: graveyard slot, early morning, morning, daytime, early fringe, prime access, primetime,
and late fringe. Group viewings are split into a training set (1,210,316 entries), and a test set
(238,748 entries) with a 80%-20% ratio. Results are reported in Table 1. Note that the superiority
of our method, recall-wise, is very pronounced. For what regards the ethical guarantees, FARGO,
delivers a very good score disparity, while, for what regards the recommendation disparity, it
seems to perform generally worse than the other methods (except for  = 1, for which its
performance is on par with the other methods). Note that for GFAR it is not possible to compute
the score disparity as it does not involve the computation of group scores for the items.</p>
      </sec>
      <sec id="sec-4-2">
        <title>4.2. Music Dataset</title>
        <p>This dataset1 has been created within the scope of a user study by asking participants to fill in
two diferent forms: an individual form collecting demographic data (i.e., age and gender) and
contextual individual preferences about music artists, and a group form to be filled in groups
asking for a collective choice of a music artist that was available at the time of the choice in a
particular context. The following two listening contexts have been selected, considering that
both are common situations users can relate to both when alone and when with other people,
and that users’ preferences would likely be diferent in each of them: during a car trip and at
dinner as background music. The dataset obtained contains data gathered from 280 users. For
each user, preferences regarding both the car trip and dinner contexts are gathered. From the
group forms, 498 context-aware collective preferences have been gathered. Of this, 272 groups
were composed of 2 users, 158 of 3 users, 32 of 4 users and 36 of 5 users. As for the previous
dataset, we used a 80%-20% split for training and test sets. Results are reported in Table 2. Also
in this case FARGO delivers the best recall. Contrarily to the previous dataset, in this case our
method achieves a very good recommendation disparity. For what regards the score disparity, all
methods provide very low (i.e., good) values.</p>
        <p>1The dataset can be downloaded at https://github.com/azzada/FARGO.</p>
        <p>Recall
25.00%
12.50%
11.11%
13.19%
12.50%
22.92%
13.89%
6.06%</p>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>5. Conclusions</title>
      <p>In this paper we have introduced FARGO, a new method for providing fair, context-aware
recommendations to ephemeral groups able also to recommend items new in the system.
Considering both recall and fairness, it is not possible to identify a best overall method across all
datasets and values of . Even if we ignored recall, a clear winner fairness-wise is not evident
(all methods tested, except for Dis, perform best fairness-wise for at least a value of  in at least
one dataset). We argue that the relationship between fairness and recommendation accuracy
should be seen as a tradeof. On both datasets of our experiments, FARGO provides the best
solution to such tradeof by achieving the best recall across all values of  while delivering
similar ethical guarantees to the other fair methods tested. Contrarily to what one might think,
LM is not the best method fairness-wise, and this implies that the problem of maximizing
both recall and fairness is not a simple one. This is a complex problem that deserves further
investigations, as recall and fairness seem not to be inversely correlated in a trivial manner.
[8] J. Masthof, Group modeling: Selecting a sequence of television items to suit a group of
viewers, in: Personalized Digital Television, Springer, 2004, pp. 93–141.
[9] E. Ntoutsi, K. Stefanidis, K. Nørvåg, H.-P. Kriegel, Fast group recommendations by applying
user clustering, in: Proc. ER, 2012, pp. 126–140.
[10] A. J. Chaney, M. Gartrell, J. M. Hofman, J. Guiver, N. Koenigstein, P. Kohli, U. Paquet, A
large-scale exploration of group viewing patterns, in: Proc. TVX, 2014, pp. 31–38.
[11] T. De Pessemier, S. Dooms, L. Martens, Comparison of group recommendation algorithms,</p>
      <p>Multimedia Tools Appl. 72 (2014) 2497–2541.
[12] N.-r. Kim, J.-H. Lee, Group recommendation system: Focusing on home group user in tv
domain, in: Proc. SCIS, 2014, pp. 985–988.
[13] I. Ali, S.-W. Kim, Group recommendations: approaches and evaluation, in: Proc. IMCOM,
2015, pp. 1–6.
[14] M. Gartrell, X. Xing, Q. Lv, A. Beach, R. Han, S. Mishra, K. Seada, Enhancing group
recommendation by incorporating social relationship interactions, in: Proc. GROUP, 2010,
pp. 97–106.
[15] S. Berkovsky, J. Freyne, Group-based recipe recommendations: analysis of data aggregation
strategies, in: Proc. RecSys, 2010, pp. 111–118.
[16] S. Yao, B. Huang, New fairness metrics for recommendation that embrace diferences,</p>
      <p>CoRR abs/1706.09838 (2017).
[17] Y. Li, Y. Ge, Y. Zhang, Tutorial on fairness of machine learning in recommender systems,
in: Proc. SIGIR, 2021, pp. 2654–2657.
[18] L. Baltrunas, T. Makcinskas, F. Ricci, Group recommendations with rank aggregation and
collaborative filtering, in: Proc. RecSys, 2010, pp. 119–126.
[19] C. Senot, D. Kostadinov, M. Bouzid, J. Picault, A. Aghasaryan, C. Bernier, Analysis of
strategies for building group profiles, in: Proc. UMAP, 2010, pp. 40–51.
[20] J. Gorla, N. Lathia, S. Robertson, J. Wang, Probabilistic group recommendation via
information matching, in: Proc. WWW, 2013, pp. 495–504.
[21] S. Ghazarian, M. A. Nematbakhsh, Enhancing memory-based collaborative filtering for
group recommender systems, Expert Syst. Appl. 42 (2015) 3801–3812.
[22] S. Amer-Yahia, S. B. Roy, A. Chawlat, G. Das, C. Yu, Group recommendation: Semantics
and eficiency, in: Proc. VLDB, 2009, pp. 754–765.
[23] Z. Yu, X. Zhou, Y. Hao, J. Gu, Tv program recommendation for multiple viewers based on
user profile merging, User Model. User-Adapt. Int. 16 (2006) 63–82.
[24] L. Xiao, Z. Min, Z. Yongfeng, G. Zhaoquan, L. Yiqun, M. Shaoping, Fairness-aware group
recommendation with pareto-eficiency, in: Proc. RecSys, 2017, pp. 107–115.
[25] S. Qi, N. Mamoulis, E. Pitoura, P. Tsaparas, Recommending packages to groups, in: Proc.</p>
      <p>ICDM, 2016, pp. 449–458.
[26] D. Serbos, S. Qi, N. Mamoulis, E. Pitoura, P. Tsaparas, Fairness in package-to-group
recommendations, in: Proc. WWW, 2017, pp. 371–379.
[27] M. Kaya, D. Bridge, N. Tintarev, Ensuring fairness in group recommendations by
ranksensitive balancing of relevance, in: Proc. RecSys, 2020, pp. 101–110.
[28] J. Leonhardt, A. Anand, M. Khosla, User fairness in recommender systems, in: Proc.</p>
      <p>WWW, 2018, pp. 101–102.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>F.</given-names>
            <surname>Ricci</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Rolach</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Shapira</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P. B.</given-names>
            <surname>Kantor</surname>
          </string-name>
          , Recommender Systems Handbook, Springer,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>E.</given-names>
            <surname>Quintarelli</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Rabosio</surname>
          </string-name>
          , L. Tanca,
          <article-title>Recommending new items to ephemeral groups using contextual user influence</article-title>
          ,
          <source>in: Proc. RecSys</source>
          ,
          <year>2016</year>
          , pp.
          <fpage>285</fpage>
          -
          <lpage>292</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>E.</given-names>
            <surname>Quintarelli</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Rabosio</surname>
          </string-name>
          , L. Tanca,
          <article-title>Eficiently using contextual influence to recommend new items to ephemeral groups</article-title>
          ,
          <source>Inf. Syst</source>
          .
          <volume>84</volume>
          (
          <year>2019</year>
          )
          <fpage>197</fpage>
          -
          <lpage>213</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>G.</given-names>
            <surname>Adomavicius</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Tuzhilin</surname>
          </string-name>
          ,
          <string-name>
            <surname>Context-Aware Recommender</surname>
            <given-names>Systems</given-names>
          </string-name>
          , Springer,
          <year>2011</year>
          , pp.
          <fpage>217</fpage>
          -
          <lpage>253</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>K.</given-names>
            <surname>Verbert</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Manouselis</surname>
          </string-name>
          ,
          <string-name>
            <given-names>X.</given-names>
            <surname>Ochoa</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Wolpers</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Drachsler</surname>
          </string-name>
          ,
          <string-name>
            <given-names>I.</given-names>
            <surname>Bosnic</surname>
          </string-name>
          , E. Duval,
          <article-title>Contextaware recommender systems for learning: A survey and future challenges</article-title>
          ,
          <source>IEEE Transactions on Learning Technologies</source>
          <volume>5</volume>
          (
          <year>2012</year>
          )
          <fpage>318</fpage>
          -
          <lpage>335</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>J.</given-names>
            <surname>Masthof</surname>
          </string-name>
          , Group Recommender Systems: Combining Individual Models, Springer,
          <year>2011</year>
          , pp.
          <fpage>677</fpage>
          -
          <lpage>702</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <surname>M. O'Connor</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          <string-name>
            <surname>Cosley</surname>
            ,
            <given-names>J. A.</given-names>
          </string-name>
          <string-name>
            <surname>Konstan</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          <string-name>
            <surname>Riedl</surname>
          </string-name>
          ,
          <article-title>Polylens: A recommender system for groups of users</article-title>
          ,
          <source>in: Proc. ECSCW</source>
          ,
          <year>2001</year>
          , pp.
          <fpage>199</fpage>
          -
          <lpage>218</lpage>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>