=Paper=
{{Paper
|id=Vol-2440/AUXPAPER
|storemode=property
|title=None
|pdfUrl=https://ceur-ws.org/Vol-2440/AUXPAPER.pdf
|volume=Vol-2440
|dblpUrl=https://dblp.org/rec/conf/recsys/BurkeAMTZ19a
}}
==None==
https://ceur-ws.org/Vol-2440/AUXPAPER.pdf
RMSE: Workshop on Recommendation in Multistakeholder
Environments∗
Robin Burke Himan Abdollahpouri Edward C. Malthouse
Dept. of Information Science Dept. of Information Science Spiegel Research Center
University of Colorado University of Colorado Northwestern University
Boulder, Colorado Boulder, Colorado Evanston, Illinois
KP Thai Yongfeng Zhang
Squirrel AI Learning Dept. of Computer Science
Shanghai, China Rutgers University
New Brunswick, New Jersey
ABSTRACT considerations of the retailer will also enter into the recommen-
In research practice, recommender systems are typically evaluated dation function. A business may wish to highlight products that
on their ability to provide items that satisfy the needs and interests are more profitable or that are currently on sale, for example. Com-
of the end user. However, in many recommendation domains, the mercial recommender systems often use separate “business rules”
user for whom recommendations are generated is not the only functionality to integrate such items into the personalized recom-
stakeholder in the recommendation outcome. For example, fairness mendations generated through conventional means. Adding the
and balance across stakeholders is important in some recommenda- retailer as a stakeholder allows such considerations to be integrated
tion applications; achieving a goal such as promoting new sellers in throughout the recommendation process.
a marketplace might be important in others. Such multistakeholder The workshop encouraged submissions addressing the chal-
environments present unique challenges for recommender system lenges of producing recommendations in multistakeholder settings,
design and evaluation, and these challenges were the focus of this including but not limited to the following topics:
workshop. • The requirements of different multistakeholder applications
such as:
CCS CONCEPTS – Recommendation in multisided platforms
• Information systems → Evaluation of retrieval results; • Social – Fairness-aware recommendation
and professional topics → User characteristics. – Multi-objective optimization in Recommendation
– Value-aware recommendation in commercial settings
KEYWORDS – Reciprocal recommendation
• Algorithms for multistakeholder recommendation including
multistakeholder recommendation; fairness; discrimination; bias;
multi-objective optimization, re-ranking and others.
e-commerce
• The evaluation of multistakeholder recommendation sys-
tems.
1 WORKSHOP TOPIC • User experience considerations in multistakeholder recom-
One of the defining characteristics of recommender systems is per- mendation including ethics, transparency, and interfaces for
sonalization. Recommender systems are typically evaluated on their different stakeholders.
ability to provide items that satisfy the needs and interests of the The RMSE 2019 workshop is a continuation of the discussion of
end user. However, in many recommendation domains, the user these topics in prior RecSys workshops including Value-Aware and
for whom recommendations are generated is not the only stake- Multistakeholder Recommendation (VAMS 2017) [3] and Responsi-
holder in the recommendation outcome. Other users, the providers ble Recommendation (FATRec 2017 and 2018) [6, 7].
of products, and even the system’s own objectives may need to
be considered when these perspectives differ from those of end
users. Fairness and balance are important examples of system-level
2 WORKSHOP ORGANIZATION
objectives, and these social-welfare-oriented goals may at times The program committee for the workshop included (in addition to
run counter to individual preferences. Sole focus on the end user the organizers listed above):
hampers developers’ ability to incorporate such objectives into • Gediminas Adomavicius, University of Minnesota
recommendation algorithms and system designs. • James Caverlee, Texas A&M University
In addition, in many e-commerce retail settings, recommenda- • Dietmar Jannach, University of Klagenfurt
tion is viewed as a form of marketing and, as such, the economic • Toshihiro Kamishima, National Institute of Advanced Indus-
trial Science and Technology (AIST)
∗ Copyright 2019 for this paper by its authors. Use permitted under Creative Commons
• Rishabh Mehrotra, Spotify Research
License Attribution 4.0 International (CC BY 4.0).
Presented at the RMSE workshop held in conjunction with the 13th ACM Conference • Nasim Sonboli, University of Colorado Boulder
on Recommender Systems (RecSys), 2019, in Copenhagen, Denmark. • Yong Zheng: Illinois Institute of Technology
RecSys ’19, September 16–20, 2019, Copenhagen, Denmark Robin Burke, Himan Abdollahpouri, Edward C. Malthouse, KP Thai, and Yongfeng Zhang
• Morteza Zihayat, Ryerson University for this optimization problem. The last paper was presented
by [5] where authors investigated the simplification of the
3 WORKSHOP PROGRAM objective function in Groupon’s search and recommenda-
tion system which is two-sided marketplace to capture the
The workshop schedule included a keynote address by Ed Chi of
essence of short, mid and long term benefits while preserving
Google, and three paper sessions featuring 10 papers (6 long, 4
fairness and moving users forward in the customer lifecycle.
short) in the following areas:
Additional information about the workshop can be found at the
• Fairness: defining, evaluating, and implementing different following URL: https://sites.google.com/view/rmse
aspects of fairness for recommender systems. In this session,
three papers were presented and discussed. In [4], Deldjoo REFERENCES
et al. introduced a fairness measure based on the generalized [1] Himan Abdollahpouri and Robin Burke. 2019. Multi-stakeholder Recommenda-
cross entropy to ensure the outcome of the recommendations tion and its Connection to Multi-sided Fairness. In Workshop on Recommendation
in Multi-stakeholder Environments (RMSE’19), in conjunction with the 13th ACM
matches a pre-defined utility for each group. Abdollahpouri Conference on Recommender Systems, RecSys’19, 2019.
and Burke [1] developed a taxonomy of different types of [2] Himan Abdollahpouri, Masoud Mansoury, Robin Burke, and Bamshad Mobasher.
multi-stakeholder recommender systems based on the archi- 2019. The Unfairness of Popularity Bias in Recommendation. In Workshop on
Recommendation in Multi-stakeholder Environments (RMSE’19), in conjunction
tecture of the system. They categorized such systems into with the 13th ACM Conference on Recommender Systems, RecSys’19, 2019.
1) multi-receiver recommenders, 2) multiprovider recom- [3] Robin Burke, Gediminas Adomavicius, Ido Guy, Jan Krasnodebski, Luiz Pizzato,
Yi Zhang, and Himan Abdollahpouri. 2017. VAMS 2017: Workshop on Value-
menders and 3) recommenders with side stakeholders. The Aware and Multistakeholder Recommendation. In Proceedings of the Eleventh
authors also showed the close connection between multi- ACM Conference on Recommender Systems. ACM, 378–379.
stakeholder recommendation and multi-sided fairness. In [4] Yahshar Deldjoo, Vito Walter Anelli, Hamed Zamani, Alejandro Bellogin Kouki,
and Tommaso Di Noia. 2019. Recommender Systems Fairness Evaluation via
[12], Strucks et al. investigated BlurMe, a gender obfusca- Generalized Cross Entropy. In Workshop on Recommendation in Multi-stakeholder
tion technique that has been shown to block classifiers from Environments (RMSE’19), in conjunction with the 13th ACM Conference on Recom-
inferring binary gender from users’ profiles and proposed mender Systems, RecSys’19, 2019.
[5] Joaquin Delgado, Samuel Lind, Carl Radecke, and Satish Konijeti. 2019. Simple
an extension to BlurMe, called BlurM(or)e, that addresses Objectives Work Better. In Workshop on Recommendation in Multi-stakeholder
the privacy issues associated with BlurMe. Environments (RMSE’19), in conjunction with the 13th ACM Conference on Recom-
mender Systems, RecSys’19, 2019.
• Calibration: matching recommendation output to user pref- [6] Michael D Ekstrand and Amit Sharma. 2017. FATREC Workshop on Responsible
erences and examining disparities between types of users Recommendation. In Proc. ACM RecSys ’18. ACM, 382–383. https://doi.org/10.
and types of items. There were four papers presented in this 1145/3109859.3109960
[7] Toshihiro Kamishima, Pierre-Nicolas Schwab, and Michael D Ekstrand. 2018. 2nd
session. Tsintzou et al. [13] proposed a metric called bias FATREC workshop: responsible recommendation. In Proc. ACM RecSys ’18. ACM,
disparity to measure the difference between the bias towards 516–516. https://doi.org/10.1145/3240323.3240335
different movie genres in user profiles and in recommenda- [8] Kun Lin, Nasim Sonboli, Bamshad Mobasher, and Robin Burke. 2019. Crank up
the volume: preference bias amplification in collaborative recommendation. In
tions. Some of the other papers in this session were inspired Workshop on Recommendation in Multi-stakeholder Environments (RMSE’19), in
by this work. Mansoury et al. [11] explored how different conjunction with the 13th ACM Conference on Recommender Systems, RecSys’19,
2019.
recommendation algorithms reflect the trade-off between [9] Raphael Louca, Moumita Bhattacharya, Diane Hu, and Liangjie Hong. 2019.
ranking quality and bias disparity. Their experiments in- Joint Optimization of Profit and Relevance for Recommendation Systems in E-
cluded neighborhood-based, model-based, and trust-aware commerce. In Workshop on Recommendation in Multi-stakeholder Environments
(RMSE’19), in conjunction with the 13th ACM Conference on Recommender Systems,
recommendation algorithms. Another paper in this session RecSys’19, 2019.
was a work by Lin et al. [8] where authors examined bias dis- [10] Edward Malthouse, Khadija Ali Vakeel, Yasaman Hessary Kamyab, Robin Burke,
parity over a range of different algorithms and for different and Morana Fuduric. 2019. A Multistakeholder Recommender Systems Algorithm
for Allocating Sponsored Recommendations. In Workshop on Recommendation
item categories and demonstrate significant differences be- in Multi-stakeholder Environments (RMSE’19), in conjunction with the 13th ACM
tween model-based and memory-based algorithms. Finally, Conference on Recommender Systems, RecSys’19, 2019.
[11] Masoud Mansoury, Bamshad Mobasher, Robin Burke, and Mykola Pechenizkiy.
Abdollahpouri et al. in [2] investigated the unfairness of 2019. Bias Disparity in Collaborative Recommendation: Algorithmic Evaluation
popularity bias in recommnder systems and how it affects and Comparison. In Workshop on Recommendation in Multi-stakeholder Environ-
different user groups differently. They show that in many ments (RMSE’19), in conjunction with the 13th ACM Conference on Recommender
Systems, RecSys’19, 2019.
recommendation algorithms the recommendations the users [12] Christopher Strucks, Manel Slokom, and Martha Larson. 2019. BlurM(or)e: Revisit-
get are extremely concentrated on popular items even if a ing Gender Obfuscation in the User-Item Matrix. In Workshop on Recommendation
user is interested in long-tail and non-popular items showing in Multi-stakeholder Environments (RMSE’19), in conjunction with the 13th ACM
Conference on Recommender Systems, RecSys’19, 2019.
an extreme bias disparity. [13] Virginia Tsintzou, Evaggelia Pitoura, and Panayiotis Tsaparas. 2019. Bias Dis-
• Multistakeholder Recommendation: implementing rec- parity in Recommendation Systems. In Workshop on Recommendation in Multi-
stakeholder Environments (RMSE’19), in conjunction with the 13th ACM Conference
ommender systems that balance interests of users and those on Recommender Systems, RecSys’19, 2019.
of other stakeholders. There were three papers presented in
this session. Louca et al. [9] presented an approach to jointly
optimize for relevance and profit. Another paper regarding
the optimization of revenue while keeping an acceptable
level of accuracy for allocating sponsored content in recom-
mendation was presented by Malthouse et al. [10] where they
used a multi-objective binary integer programming model