=Paper=
{{Paper
|id=Vol-2440/paper3
|storemode=property
|title=Multi-stakeholder Recommendation and its Connection to Multi-sided Fairness
|pdfUrl=https://ceur-ws.org/Vol-2440/paper3.pdf
|volume=Vol-2440
|authors=Himan Abdollahpouri,Robin Burke
|dblpUrl=https://dblp.org/rec/conf/recsys/AbdollahpouriB19
}}
==Multi-stakeholder Recommendation and its Connection to Multi-sided Fairness==
https://ceur-ws.org/Vol-2440/paper3.pdf
Multi-stakeholder Recommendation and its Connection to
Multi-sided Fairness∗
Himan Abdollahpouri Robin Burke
University of Colorado Boulder, USA University of Colorado Boulder, USA
himan.abdollahpouri@colorado.edu robin.burke@colorado.edu
ABSTRACT
There is growing research interest in recommendation as a multi-
stakeholder problem, one where the interests of multiple parties
should be taken into account. This category subsumes some exist-
ing well-established areas of recommendation research including
reciprocal and group recommendation, but a detailed taxonomy
of different classes of multi-stakeholder recommender systems is
still lacking. Fairness-aware recommendation has also grown as
a research area, but its close connection with multi-stakeholder
recommendation is not always recognized. In this paper, we define
the most commonly observed classes of multi-stakeholder recom-
mender systems and discuss how different fairness concerns may
Figure 1: A multi-stakeholder problem at Uber Eats 2
come into play in such systems.
KEYWORDS discussed in [24]. One of its accepted definitions is to avoid giving
Recommender systems; Fairness in recommendation; Multi-stakeholder discriminatory recommendations to different users based on some
recommendation; Multi-sided fairness; sensitive features such as gender or race [12]. Talking about the fair-
ness of a recommendation can be meaningful only when there are
1 INTRODUCTION different stakeholders involved. Thus, there is a close connection
between the requirements of multi-stakeholder recommendation
Recommender systems (RS) have been used in a variety of differ- and those of fairness-aware recommendation.
ent domains to help users find relevant and interesting items. They In this paper, we present three most commonly observed classes
recommend movies to watch [14] , songs to listen [9] , jobs to take of multi-stakeholder recommendation:
[17] or even a person to date [19]. • Multi-receiver recommendation
The research in RS has been mainly focused on the personaliza- • Multi-provider recommendation
tion. That is, delivering the recommendations that best match the • Recommendation with side stakeholders
needs and interests of the end user. That is indeed an important
In addition, we discuss the fairness concerns that could come
consideration as users are one of the most important stakeholders
into play in these types of recommendation systems with regard to
in any recommendation platform but not the only one [1]. There are
different stakeholders such as users, item providers, etc.
numerous examples of recommender systems in which there are
other stakeholders that their needs and preferences should be taken
2 MULTI-STAKEHOLDER
into account. The incorporation of the objectives and preferences
of different stakeholders in the recommendation process is referred RECOMMENDATION
to as multi-stakeholder recommendation [7] Recommender systems are often multi-stakeholder environments
Multi-stakeholder recommendation is a relatively new topic (at [2, 7]: there are several stakeholders (often with conflicting prefer-
least in academia) and, therefore, there is still no clear understand- ences) whose needs and preferences should be taken into account
ing of what type of recommender system is a multi-stakeholder in generating the recommendations. For example, in a food deliv-
one. Defining different classes of multi-stakeholder recommenda- ery system like the one for Uber Eats, we can observe three major
tion would help to distinguish among different recommendation stakeholders as shown in Figure 1: 1) the eaters or otherwise users
problems ,which is important for developing the right algorith- who use the app and receive the recommendations for different
mic solutions for these types of recommender systems. Moreover, restaurants, 2) the restaurant that are being recommended and 3)
fairness-aware recommendation as a growing sub-topic in recom- the delivery partners that take the food from the restaurants to
mendation research has gained a lot of attention in recent years. the user’s address. Eaters want to get relevant and interesting rec-
Fairness in recommendation can be defined in different ways, as ommendations; restaurants want to be given a fair exposure to
∗ Copyright 2019 for this paper by its authors. Use permitted under Creative Commons
different users so they can have enough customers and finally the
License Attribution 4.0 International (CC BY 4.0). delivery partners need to be satisfied with the type of orders they
Presented at the RMSE workshop held in conjunction with the 13th ACM Conference need to deliver. Uber Eats cannot survive without the existence of
on Recommender Systems (RecSys), 2019, in Copenhagen, Denmark.
RMSE workshop at ACM RecSys 2019, September 20, Copenhagen, Denmark. 2 https://eng.uber.com/uber-eats-recommending-marketplace/
any of these three parties and therefore it needs to take all of these they somehow involve in the challenges and requirements
stakeholders’ preferences into account. This example is clearly be- that comes with those recommended course (e.g. paying
yond the standard user-focused recommender system in which the the fee for the registration etc.) so their preferences should
needs and preferences of the end user is the only consideration. be also taken into account in recommending those courses.
Although the research in recommender system has been mainly Zheng et al. [27] proposed a utility-based multi-stakeholder
focused on the satisfaction of the end user, there exist several recommendations to the area of personalized learning in ed-
threads of research in the recommender system that are exam- ucations in which the preferences of students and instructors
ples of multi-stakeholder recommendation but similarities among are taken into account simultaneously. Figure 2-a shows a
them have not typically been recognized. For example, in a group typical architecture for this type of systems. We showed the
recommendation platform [10] the system wants to recommend users with different icons to clarify the heterogeneousness
an item or a list of items to a group of users such that it meets the of the receivers.
preferences of the group members. That is different from recom- • Homogeneous: In contrast to the heterogeneous multi-
mending items to only one user as, in many cases, the preferences of receiver recommendation, users in homogeneous multi-receiver
the users in the group could conflict with each other and, therefore, recommendation are all from the same type (e.g. students)
it is important for the recommender system to find ways to handle and they are consuming the recommended item. This type
such conflicts. of multi-receiver recommendation is known as group recom-
Another area of work that also falls into the multi-stakeholder mendation in the literature where an item or set of items is
definition is the reciprocal recommendation where the preferences recommended to a group of users who all use or consume
of both the receiver and the provider of the recommendations the recommendation. For example, a movie recommender
should be taken into account [25]. For example in online dating system that recommends movies to a group of friends or
applications [20, 22], recommending a user to another user is only family members falls within this category as all these users
meaningful if both users are happy with this recommendation. are the real consumers of these recommendations. See the
Multi-stakeholder recommendation can be seen as an umbrella survey in [15]. Figure 2-b shows a typical architecture for
for many different types of recommendation problems involving this type of systems. The users are all shown with the same
two or more different stakeholders. These types of systems have icon.
not yet been studied in a systematic way and there is still a lot of
room for further research and development. 3.2 Multi-provider Recommendation
In next section, we define the most commonly observed classes of
It is often the case that the items and products on a recommender
multi-stakeholder recommendation and provide examples for each.
system are provided by several parties that use the platform for
We believe having a well-defined architecture for these systems
reaching out to their desired audience. For example, in a sharing
can facilitate further research in this area.
economy platform like Airbnb all the available listings on the plat-
form are actually provided by different hosts and the Airbnb itself
3 CLASSES OF MULTI-STAKEHOLDER does not own any of these apartments. These hosts are using the
RECOMMENDATION Airbnb platform to find travellers that could be interested in staying
In this section we define different classes of multi-stakeholder rec- at their room or apartment. Therefore, Airbnb is a multi-provider rec-
ommendation according to the architecture of the recommendation ommendation platform since there are numerous number of hosts in
platform. One could argue there exist more types of architectures the system and Airbnb should try to give a fair amount of exposure
but we believe these are typically the main and commonly observed to the listings provided by each of these hosts. Several interesting
patterns. challenges could arise form this type of systems such as cold start
providers (a provider that is new to the system and needs to get
3.1 Multi-receiver Recommendation attention), malicious providers (a provider who is violating some
The first class of multi-stakeholder recommendation is Multi-receiver rules or s/he has bad ratings from the users) and problems related
Recommendation where the receiver of the recommendation is not to fairness which we discuss later in section 4. Figure 3 shows
one individual but rather a group of individuals. In these systems, the architecture for this type of systems. As you can see, multiple
the recommendation should be appealing to all the individuals re- providers are giving their items to the system to be recommended
ceiving the recommendation. Depending on the characteristics of to the users. The multi-provider recommender can be seen in two
the receivers of the recommendation, there could exist two types different types depending on whether or not the providers have
of multi-receiver recommendation: preferences towards what type of users they want to reach out to:
• Heterogeneous: In this type of multi-receiver recommenda- • Without Provider Preferences In this type of multi-provider
tion, the receivers of the recommendation may not necessar- recommendation the system is aware that there are multi-
ily be the same type. For example, in an educational system ple providers behind the items and they should get a fair
that recommends courses to students, the receivers of the rec- exposure in the recommendations. However, the providers
ommendations are the students who take those courses. In themselves have no preferences towards certain users to
addition, in many cases, the parents of those students could whom their items want to be recommended. For example, on
be also considered as the receiver of the recommendation the Kiva.org microlending platform, the items which are rec-
even though they are not taking the course themselves but ommended are loans. That means the providers in this case
(a) Heterogeneous multi-receiver recommendation
(b) Homogeneous multi-receiver recommendation
Figure 2: The architectures for a) Heterogeneous and b) Homogeneous multi-receiver recommendations
Figure 3: Multi-provider recommender system
are the people who asked for loan on the platform (i.e. the • With Provider Preferences The providers in this case ac-
borrowers). In this example, the borrowers do not really care tually have certain type of audience in mind and the rec-
to whom their loan requests are recommended as long as ommender system tries to fulfill the providers’ preferences.
the loan has a good chance of being funded. In other words, For example, in computational advertising an ad should be
they do not have a specific target audience in mind. recommended to a user if s/he falls within the predefined
Figure 4: Recommendation with side stakeholders
groups of users who should get that particular ad [26]. In In this model, there are some side stakeholders in addition to the re-
other words, in addition to the ad not being annoying to ceivers and item providers whose preferences should be taken into
the user (sadly it is often the case), the user should also be account by the recommender system. One interesting challenge in
acceptable from the advertiser’s perspective as they might this type of systems is how the system should aggregate those pref-
want to reach certain users with a specific characteristics to erences and to what extent each stakeholder’s preferences should
maximize their ad efficiency. be incorporated or prioritized in this decision making process.
It is worth mentioning that, in some literature, the multi-provider 3.3.1 Value-aware recommendation. A frequent instantiation of a
recommendation is referred to as two-sided market [16] where on recommendation with side stakeholders is known as value-aware
one side we have the consumers and on the other side we have recommendation [5, 18] where the recommendation platform is also
the item providers. We believe the name multi-provider recom- considered as a stakeholder as it may have some goals and prefer-
mendation can better explain the behavior of these type of recom- ences with respect to the recommendations–considerations such as
mendation systems as it clearly emphasizes the other side of the profit maximization [4, 11], long-tail promotion [3, 21] are examples
recommendation (besides the consumers): the item providers. of such goals and preferences. These systems are called value-aware
recommendation because the recommender system needs to make
sure, in addition to the users’ satisfaction, the recommendations
3.3 Recommendation with Side Stakeholders bring some sort of value to the business. For example, authors in
Stakeholders in a recommender system may not be always a direct [11] proposed a value-aware movie recommendation that takes
part of the recommendation interaction. In other words they do both the price for each movie and also the user satisfaction into
not have to necessarily be the consumer of recommendations nor account in generating the recommendations. They showed it was
do they have to be the provider of the items being recommended. possible to significantly increase the revenue with a negligible loss
In some recommendation problems, there are other parties that in accuracy.
are being affected by a given recommendation. In such cases, the
satisfaction of these side stakeholders should be taken into account 3.4 Hybrid
in recommending items to users. For instance, on Uber Eats, when a These were the most commonly observed classes of multi-stakeholder
list of restaurants is being recommended to a user, the person who is recommendation. One can imagine a combination of any of these
responsible for delivering food to the user is also potentially affected classes. For example, the Uber Eats platform is a combination of
by this recommendation. Not only are the driver’s preferences multi-provider recommender system (different restaurants) and
for a given recommendation important, the overall economy of recommender with side stakeholders (the delivery partners). In fact,
the amount of work load given to the delivery partners is also Uber Eats could be also a value-aware recommender system as the
something that the system should consider. Uber Eats itself might also have certain business goals in mind.
The main characteristics of this type of recommendation sys-
tems is the fact that other stakeholders who are affected by the
recommendations are not the entities who provide the recommen-
4 FAIRNESS IN MULTI-STAKEHOLDER
dations (multi-provider) nor are they the users who receive the RECOMMENDATION
recommendations (multi-receiver). As in the Uber Eats example, Recently, fairness-aware recommendation has attracted a lot of
the delivery partners are not receiving food recommendations nor attention from the recommender systems research community
are they providing the recommendations (like restaurants) but they [6, 13, 24]. As we discussed earlier, there is a very close connection
still are affected by these recommendations. Figure 4 shows the between multi-stakeholder recommendation and fairness-aware
architecture of a recommendation platform with side stakeholders. recommendation as having multiple stakeholders gives rise to the
questions of fairness in recommendation. As in [1, 8] we can see several different types of fairness that are important to address in a
different classes of fairness, distinguished by the fairness issues that multi-stakeholder recommendation. For future work, we intend to
arise relative to what authors call different sides of the platform: develop algorithmic solutions for each of these different types of
consumers (C-fairness) and providers (P-fairness). In this paper, as multi-stakeholder recommendation.
we saw in section 3.4, there could be some stakeholders that are nei-
ther the receiver nor are they the provider of the recommendations.
Therefore, in addition to the mentioned fairness types, we define REFERENCES
another type which accounts for the fairness towards other affected [1] Himan Abdollahpouri, Gediminas Adomavicius, Robin Burke, Ido Guy, Dietmar
stakeholders in the system. We call this type of fairness S-Fairness Jannach, Toshihiro Kamishima, Jan Krasnodebski, and Luiz Pizzato. 2019. Beyond
which refers to the idea of the recommender system being fair to Personalization: Research Directions in Multistakeholder Recommendation. arXiv
preprint arXiv:1905.01986 (2019).
the side stakeholders who are not directly participating in the rec- [2] Himan Abdollahpouri, Robin Burke, and Bamshad Mobasher. 2017. Recommender
ommendation process but rather being affected by such transaction. systems as multi-stakeholder environments. In Proceedings of the 25th Conference
on User Modeling, Adaptation and Personalization (UMAP2017). ACM.
Therefore, in a multi-stakeholder recommendation platform we can [3] Himan Abdollahpouri, Robin Burke, and Bamshad Mobasher. 2019. Managing
observe the following types of fairness: Popularity Bias in Recommender Systems with Personalized Re-ranking.. In
Florida AI Research Symposium (FLAIRS). ACM, To appear.
• C-fairness: A recommender system distinguished by C- [4] Amos Azaria, Avinatan Hassidim, Sarit Kraus, Adi Eshkol, Ofer Weintraub, and
fairness is one that must take into account the disparate Irit Netanely. 2013. Movie recommender system for profit maximization. In
Proceedings of the 7th ACM conference on Recommender systems. ACM, 121–128.
impact of recommendation on protected classes of recom- [5] Robin Burke, Gediminas Adomavicius, Ido Guy, Jan Krasnodebski, Luiz Pizzato,
mendation consumers. For example, in a job recommender Yi Zhang, and Himan Abdollahpouri. 2017. VAMS 2017: Workshop on Value-
system that has a C-fairness requirement, the recommenda- Aware and Multistakeholder Recommendation. In Proceedings of the Eleventh
ACM Conference on Recommender Systems. ACM, 378–379.
tions should be fair towards the users in the protected class [6] Robin Burke, Nasim Sonboli, Masoud Mansoury, and Aldo Ordoñez-Gauger. 2017.
(as defined by gender, age, nationality, etc.) relative to other Balanced Neighborhoods for Fairness-aware Collaborative Recommendation. In
users. Workshop on Responsible Recommendation (FATRec).
[7] Robin D. Burke, Himan Abdollahpouri, Bamshad Mobasher, and Trinadh Gupta.
In this paper, however, we extend the C-fairness to any fair- 2016. Towards Multi-Stakeholder Utility Evaluation of Recommender Systems.
ness concerns that the system might have with respect to In Workshop on Surprise, Opposition, and Obstruction in Adaptive and Personalized
the consumers of the recommendation. For instance, in a Systems, UMAP 2016.
[8] Burke, Robin. 2017. Multisided Fairness for Recommendation. In Workshop on
group recommender system the C-fairness could refer to Fairness, Accountability and Transparency in Machine Learning (FATML). Halifax,
taking into account the preferences of different members Nova Scotia, 5 pages.
[9] Oscar Celma. 2010. Music recommendation. In Music recommendation and
of the group in a fair way and avoid ignoring certain users’ discovery. Springer, 43–85.
preferences for a long term [23]. [10] Alexander Felfernig, Ludovico Boratto, Martin Stettinger, and Marko Tkalčič.
• P-fairness: A recommender system that has a P-fairness 2018. Group recommender systems: An introduction. Springer.
[11] Dietmar Jannach and Gediminas Adomavicius. 2017. Price and Profit Awareness
requirement should treat the providers of the items in a in Recommender Systems. In Proceedings of the ACM RecSys 2017 Workshop on
fair way. For example, in job recommendation platform, it Value-Aware and Multi-Stakeholder Recommendation. Como, Italy.
[12] Toshihiro Kamishima, Shotaro Akaho, Hideki Asoh, and Jun Sakuma. 2012. En-
could mean that minority-owned businesses have their jobs hancement of the Neutrality in Recommendation.. In Decisions@ RecSys. 8–14.
recommended to qualified candidates. [13] Eric L. Lee, Jing-Kai Lou, Wei-Ming Chen, Yen-Chi Chen, Shou-De Lin, Yen-Sheng
• S-Fairness There are examples of multi-stakeholder recom- Chiang, and Kuan-Ta Chen. 2014. Fairness-Aware Loan Recommendation for
Microfinance Services. ACM Press, 1–4.
mendation in which some stakeholder are not the receiver [14] George Lekakos and Petros Caravelas. 2008. A hybrid approach for movie
nor are they the providers of the recommendations. Instead, recommendation. Multimedia tools and applications 36, 1-2 (2008), 55–70.
they act as side stakeholders whose needs and preferences [15] Judith Masthoff. 2011. Group recommender systems: Combining individual
models. In Recommender systems handbook. Springer, 677–702.
should be taken into account in a fair way. For example, in [16] Rishabh Mehrotra, James McInerney, Hugues Bouchard, Mounia Lalmas, and
the Uber Eats example, the delivery partners are side stake- Fernando Diaz. 2018. Towards a fair marketplace: Counterfactual evaluation
of the trade-off between relevance, fairness & satisfaction in recommendation
holders and, therefore, the system needs to generate the systems. In Proceedings of the 27th ACM International Conference on Information
recommendations in a way that the fairness considerations and Knowledge Management. ACM, 2243–2251.
towards these delivery partners such as fair work load, fair [17] Ioannis Paparrizos, B Barla Cambazoglu, and Aristides Gionis. 2011. Machine
learned job recommendation. In Proceedings of the fifth ACM Conference on
commute distance etc. are being addressed properly. Recommender Systems. ACM, 325–328.
In particular applications, multiple fairness concerns may arise [18] Changhua Pei, Xinru Yang, Qing Cui, Xiao Lin, Fei Sun, Peng Jiang, Wenwu Ou,
and Yongfeng Zhang. 2019. Value-aware Recommendation based on Reinforced
on different sides of the interaction. Thus, a system may have any Profit Maximization in E-commerce Systems. arXiv preprint arXiv:1902.00851
combination of these fairness considerations in play at once: CP- (2019).
[19] Luiz Pizzato, Tomek Rej, Thomas Chung, Irena Koprinska, and Judy Kay. 2010.
fairness, for both consumers and providers, as noted in [1], but also RECON: a reciprocal recommender for online dating. In Proceedings of the fourth
any of the others. ACM conference on Recommender systems. ACM, 207–214.
[20] Luiz Pizzato, Tomek Rej, Thomas Chung, Irena Koprinska, and Judy Kay. 2010.
RECON: a reciprocal recommender for online dating. In Proceedings of the fourth
5 CONCLUSION AND FUTURE WORK ACM conference on Recommender systems. ACM, 207–214.
[21] Idan Szpektor, Aristides Gionis, and Yoelle Maarek. 2011. Improving recommen-
In this paper we defined several most commonly observed classes dation for long-tail queries via templates. In Proceedings of the 20th international
of multi-stakeholder recommendation. We argued how some of conference on World wide web. ACM, 47–56.
the already existing recommendation problems such as group rec- [22] Peng Xia, Benyuan Liu, Yizhou Sun, and Cindy Chen. 2015. Reciprocal Rec-
ommendation System for Online Dating. In Proceedings of the 2015 IEEE/ACM
ommendation and reciprocal recommendation are also examples International Conference on Advances in Social Networks Analysis and Mining 2015.
of multi-stakeholder recommendation. In addition, we discussed ACM, 234–241.
[23] Lin Xiao, Zhang Min, Zhang Yongfeng, Gu Zhaoquan, Liu Yiqun, and Ma Shaop- [26] Weinan Zhang, Shuai Yuan, and Jun Wang. 2014. Optimal real-time bidding
ing. 2017. Fairness-aware group recommendation with pareto-efficiency. In for display advertising. In Proceedings of the 20th ACM SIGKDD international
Proceedings of the Eleventh ACM Conference on Recommender Systems. ACM, conference on Knowledge discovery and data mining. ACM, 1077–1086.
107–115. [27] Yong Zheng, Nastaran Ghane, and Milad Sabouri. 2019. Personalized Educational
[24] Sirui Yao and Bert Huang. 2017. Beyond parity: Fairness objectives for collabora- Learning with Multi-Stakeholder Optimizations. In Adjunct Proceedings of the
tive filtering. In Advances in Neural Information Processing Systems. 2921–2930. ACM conference on User Modelling, Adaptation and Personalization. ACM.
[25] Hongtao Yu, Chaoran Liu, and Fuzhi Zhang. 2011. Reciprocal recommendation
algorithm for the field of recruitment. JOURNAL OF INFORMATION &COMPU-
TATIONAL SCIENCE 8, 16 (2011), 4061–4068.