Additional Key Words and Phrases: multi-objective recommendation, Value-aware recommendation Recommender systems are software tools that are used in a variety of application domains supporting users to find relevant items, products, and services easier. Historically, the main criterion for a successful recommender system was the relevance of the recommended items to the user. In other words, the only objective for the recommendation algorithm was to learn user's preferences for diferent items and generate recommendations accordingly. However, real-world recommender systems are well beyond a simple objective and often take into account multiple objectives. Indeed, diferent objectives can be important and should be considered for generating the recommendations. These objectives can be either from the users' perspective or they could come from other stakeholders such as item providers and the ones that could be impacted by the recommendations. From the users' perspective, often multiple objectives need to be considered for generating the recommendations. For example, in restaurant recommendations, several factors should be taken into account, such as users' taste, diet restrictions, the proximity of the restaurant, and price. Each of these considerations may be important, but to varying degrees and with heterogeneity between customers. Therefore, it is crucial for a recommender system to incorporate all these diferent objectives into account when recommending restaurants to a user. Similarly, in the education domain, a student may prefer working on simpler problems to achieve higher scores. However, students need to be challenged to learn; as a result, a system that recommends practice problems should balance student preferences with utility for learning. Objectives may also come from stakeholders such as the item providers (e.g., content creators), platform owners, or even society. For example, on a music streaming service, the platform may want to balance the multiple interests of the listeners (enjoyment), artists (exposure), and the platform as a company (revenue). These types of objectives and considerations exist in many other domains including social media, transportation, news recommendation, and food recommendation. The MORS workshop encouraged submissions addressing the challenges of producing recommendations in multiobjective and multi-stakeholder settings, including but not limited to the following topics: • Recommender systems with multiple objectives • Value-aware recommendation (profit, value, purpose, etc.)
∗Copyright 2021 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). Presented at the MORS workshop held in conjunction with the 15th ACM Conference on Recommender Systems (RecSys), 2021, in Amsterdam, Netherlands. • Trade-of between relevance and bias in recommender systems • Recommendation with multiple stakeholders • Food recommendation with diferent objectives • Group recommender systems • Conflict handling in multi-stakeholder recommendation • Fairness-aware recommender systems • Balancing the long-term impacts of the recommendations and the users’ short term preferences • News recommendation with editorial values • Educational recommender systems with multiple, potentially conflicting, objectives • Personalized medicine with the diferent objectives coming from the patients and physicians
The MORS 2021 workshop was a continuation of the discussion of these topics in prior RecSys workshops including
Value-Aware and Multistakeholder Recommendation (VAMS 2017 [
We propose a half-day workshop organized either online or hybrid (depending on conference guidelines). We expect 20 to 50 participants. For the online format of the workshop, we will use Zoom with the option of break-out rooms. Here is the description of our plan:
We will encourage the workshop participants to actively adopt the conference mobile application and share their opinions regarding the workshop. This will further boost the networking among the participants of the workshop and allow the workshop organizers to obtain some feedback from the viewpoints of the participants.
The MORS workshop’s expected outcomes can be summarized as follows: (1) Understanding various objectives and goals for recommender systems when multiple objectives, sometimes coming from multiple stakeholders are present in the system, (2) The algorithms to generate recommendations in a multi-objective, multi-stakeholder environment, and (3) Understanding new evaluation approaches when there are multiple objectives and stakeholders in recommender systems. 3
The workshop organizers were as follows:
Himan Abdollahpouri is a Research Scientist at Spotify, USA.
Mehdi Elahi is an Associate Professor at University of Bergen (UiB), Department of the Information Science & Media Studies (InfoMedia).
Masoud Mansoury is a postdoctoral fellow in Amsterdam Machine Learning Lab at University of Amsterdam.
Sherry Sahebi is an assistant professor of Computer Science at the University At Albany – SUNY and the founder of Personalized AI (PersAI) Lab.
Zahra Nazari is a senior research scientist at Spotify.
Babak Loni is Chapter Lead of Machine Learning Engineering at ING Group. 4
MORS 2021 followed a peer review process for paper acceptance. At least two program committee members reviewed each submission. The following is a list of academic and industry researchers that helped the workshop in the review process: 5
The following is a tentative timeline (based on 2020 deadlines relative to the 2021 starting date): • • First call for participation: April 8, 2021 • Paper submission deadline: August 2, 2021 • Notification of paper acceptance: August 23, 2021 • Camera-ready version deadline: September 3, 2021
Workshop (at RecSys 2021): September 25, 2021 6
The workshop starts with a keynote by Shankar Kalyanaraman titled “Measuring and mitigating long-term efects of
recommender systems: A framework and a call to action”. The workshop then follows by seven paper presentations,
consisting of two long and five short contributions. The workshop is then finalized with a panel and discussion session.
The following is the list of accepted papers:
• [
Limited Budget (long)
• [
Factor: Balancing Accuracy and Health in Recipe Recommendation Through a Nutrient-Specific Metric (short)
• [
Stakeholders (short)
• [
Momentum-based Gradient Methods in Multi-Objective Recommendation (short) 5