Recommender systems are designed to help customers in finding their personalized content. However, biases in recommender systems can potentially exacerbate over time. Multi-objective recommender system (MORS) algorithms aim to alleviate bias while maintaining the accuracy of recommendation lists. While these algorithms efectively address item-side fairness, provider-side fairness often remains neglected. This study investigates the impact of MORS algorithms, leveraging evolutionary techniques to mitigate popularity bias on the item-side, on providers' fairness. Our findings reveal that baseline algorithms can adversely afect providers' fairness. Moreover, it is demonstrated that evolutionary algorithms, specifically those introducing less popular items to the initial population of their algorithms, exhibit superior performance compared to other MORS algorithms in enhancing providers' fairness. This research sheds light on the crucial role MORS algorithms, particularly those employing evolutionary approaches, can play in mitigating bias and promoting fairness for both users and providers in recommender systems.
These days, with the increasing amount of information on
the web, content providers need systems to personalize
content for end-users. As a result, users can eficiently access
their favorite content, leading to user satisfaction [
Fairness-aware recommender systems aim to address
algorithmic bias in various ways, ensuring the system’s
recommendations are unbiased [
In RS, various stakeholders play crucial roles, with two
primary groups being consumers of items and providers of
items [
Using Multi-objective Recommender Systems (MORS) as
a post-processing approach ofers a potential solution for
achieving fairness in RS outputs [
In this study, our objective is to investigate the behavior of
MORS algorithms in mitigating item popularity bias and its
impact on providers’ fairness. While existing research has
shown the trade-of between mitigating popularity bias and
maintaining recommendation accuracy on the item side, it
is crucial to delve deeper into how existing work objectives
can influence providers’ fairness. Prior research has yet to
be conducted in this area, and our study aims to address
this gap [
Furthermore, we aim to explore which specific objectives
may have a trade-of with providers’ fairness to provide a
more comprehensive understanding of the issue. We have
chosen MORS algorithms that benefit from evolutionary
algorithms to solve a multi-objective optimization to achieve
this. While evolutionary algorithms may not be the swiftest,
their superiority in addressing multi-objective problems
arises from their capability to tackle complex and non-linear
optimization problems [
Our work shows that MORS algorithms perform better in ensuring providers’ fairness than baseline algorithms. MORS algorithms enable providers to showcase their items more efectively than baseline algorithms. Although it is noteworthy that, among all MORS algorithms, there is no significant diference in covering providers’ fairness, those algorithms that add less popular items to their initial population of evolutionary algorithms show better performance than other MORS algorithms.
The remainder of this paper is structured as follows. Section 2 reviews the related fairness in recommender systems. Section 3 describes the algorithms we use in our study and the measures we utilize to compare the algorithms’ performance. Section 4 presents some results of our framework on the MovieLens and IMDB datasets. Section 5 concludes this work.
The fundamental RS aims to forecast ratings for unknown
items among users, employing diverse algorithms for this
task. Approaches like User-based and Item-based
collaborative filtering algorithms, as explored by Adomavicius
et al. [
The investigation for an optimal balance between
accuracy and bias mitigation has garnered significant attention
in RS. Malekzadeh and Kaedi propose a strategy that
simultaneously personalizes recommended items to maintain
accuracy as discussed in their work [
Fairness-aware recommender systems try to tackle the
algorithmic bias issue in diferent ways and ensure that the
recommendations made by the system are unbiased [
In this section, our initial focus is to introduce the algorithms employed in our study. Subsequently, we will delve into the evaluation metrics utilized for comparing results. Our objective is to comprehensively explore the impact of item bias mitigation on the producers’ side fairness and understand how it influences the outcomes of the recommendation systems.
We have selected two baseline algorithms, item-based and user-based collaborative filtering, where no post-processing has been applied to the output. These algorithms serve as our baseline models for evaluating bias mitigation strategies and their impact on the producers’ side in subsequent analyses.
For computing the similarity between two users, we have: (, ) = √︁∑︀ ∈∩ ( − ).
∑︀∈∩ ( − ).( − ) √︁∑︀ ∈∩ ( − ) (1)
Equation 1 defines the similarity measure between two users, and , calculated based on the items they have both rated. Here, represents the subset of items rated by user , denotes the rating given by user to item , and is the average rating provided by user .
ˆ = + ∑︀∈() (, ).( − ) ∑︀∈() |(, )| (2)
Equation 2 outlines the predicted rating (ˆ ) of user for item . It incorporates the average rating and calculates the predicted rating by considering the similarity between user and other users () who have rated the same item (). The set () represents the group of nearest users to who have provided ratings for item . The itembased collaborative filtering is similar to the user-based collaborative filtering.
In this section, we introduce algorithms that leverage the outputs of baseline algorithms, implementing reranking strategies to achieve specific objectives. Each algorithm is characterized by an objective function to mitigate item popularity bias.
Malekzadeh and Kaedi [
Wang et al. [
=1 In this expression, denotes the length of the recommendation list. A higher 1 value signifies increased popularity of the items within the list. 2. Long tail recommendation: Items with higher ratings might be prioritized higher on the ranking list for all users, and popular items often receive similar ratings, resulting in low variance. To measure the unpopularity in terms of the mean and variance of item ratings, Tamas et al. proposed a value for an item : =
1 ( + 1)2 Here, and represent the mean and variance of ratings for item across all users. To prevent division by zero, a value of one is added to the variance. The reciprocal of this mean-variance combination yields the value , where a smaller value indicates a more popular item.
Motivated by this concept, an objective function 2 is introduced to calculate the unpopularity of the recommendation result: The attribute-based diversity measurement in this study is determined using an equation to assess a recommendation list’s diversity. The formula is expressed as:
(1, . . . , ) = 1 ( − 1) =1 = ∑︁ ∑︁(1 − similarity(, )) (4) In this context, signifies the number of items within the recommendation list, and 1, . . . , represents the items recommended. The term similarity(, ) denotes the measure of similarity between two items and based on the attribute .
Equation 3 illustrates the ideal diversity for a specific user, capturing an optimal scenario. Subsequently, the deviation between this ideal diversity and the actual diversity computed from Equation 4 for the recommendation list is measured. The disparity for each item attribute is quantified through Equation (7):
Personalized Diversity = | − Diversity| (5) In this expression, Diversity denotes the diversity of the recommendation list based on attribute , while signifies the entropy of user preferences related to attribute . This metric, termed Personalized Diversity, serves to quantify the diference between the ideal and actual diversity in the recommendation list for a given user, explicitly considering the preferences associated with a particular attribute. 2. The participation of long tail items: The long tail metric is computed using the formula: ∑︁ =1 Long Tail =
Popularity() (6) In this Equation, signifies the size of the recommendation list, representing the total number of items included in the recommendation. A lower value obtained from this calculation indicates a higher likelihood of incorporating less popular items in the recommendation list. This suggests a greater emphasis on the inclusion of long-tail items in the recommendations, reflecting a preference for diversity and coverage beyond just popular items. 3. Accuracy: The Accuracy metric is evaluated using the following Equation:
Accuracy =
1 ∑︀ =1 PredictedRate() (7) In this Equation, PredictedRate() denotes the predicted rating assigned to the item. The formula computes the inverse of the sum of the predicted ratings for all recommended items, ofering a metric to assess the accuracy of the recommendation system. A lower value in the Accuracy metric suggests a higher overall accuracy in the predicted ratings for the recommended items. (8) (9) (10) 2 = ∑︁
1 =1 ( + 1)2 This function quantifies the unpopularity of the recommended items, with lower values indicating more popular items in the list.
They employ a genetic algorithm to achieve these objectives, seeking optimal combinations of items within recommendation lists that satisfy the defined criteria. This approach aims to enhance accuracy and mitigate popularity bias for more balanced and practical recommendations.
Shafiloo et al. [
Additionally, in their research, they modified the memetic algorithm. Instead of randomly adding items to the initial population, as is common in other genetic algorithms, they introduced a higher possibility of including items from the long tail and a lower possibility of including popular items in the initial population.
() =
1 ( − 1) ∑︁ (, ) ̸=
Here, represents the length of the recommendation lists for user , and (, ) calculates the similarity between two items and based on a similarity metric defined in Equation 1. The purpose of () is to quantify the similarity of items within user ’s recommendation list.
The intra-user diversity for all users is then defined as: (14) (15) (16) (17) all users() = 1 ∑︁ () ∈
Here, denotes the number of users in the set . This Equation provides a measure of intra-user diversity considering all users in the study.
Equation 16 introduces the Normalized Discounted Cumulative Gain (NDCG) measurement, a widely used metric for evaluating the quality of recommendations. This measurement is defined as: @() @()
Here, @() represents the ideal @() for user , where the ideal scenario assumes that all relevant items in the user’s recommendation list appear at the top rank, resulting in the maximum possible @().
The discounted cumulative gain at position for user , denoted as @(), is calculated using the formula:
This section presents the datasets employed for evaluating the proposed method. Subsequently, we outline the evaluation criteria and comprehensively represent the comparison result.
In our experimental evaluation, we use 2 real-world datasets, namely MovieLens and IMDB. The MovieLens dataset is a commonly employed dataset for evaluating methods addressing long-tail problems in various studies. Specifically, we utilize the MovieLens 1M dataset that features 6040 users and 1 million ratings for 3883 items. The IMDB Dataset is also employed to enhance information about movie providers, and director information is extracted. In this study, movie directors are considered providers, and the dataset includes information on 2208 movie directors.
The study evaluates methods addressing the long-tail problem using three criteria for comparison. The first criterion is accuracy, measured through the precision metric defined as: = (11)
Here, represents the total number of items
recommended to the user, and denotes the relevant items
suggested to the user. Relevant items are those with ratings
higher than the user’s average ratings, as outlined by Wang
et al. [
The second criterion, aggregate diversity (AG) (Equation
12), counts the number of distinct items ofered to users,
particularly focusing on long-tail items in recommendation
lists [
= | ⋃︁ ()| (12) Equation 12 introduces the aggregate diversity criterion, where represents a specific user from the set of users , and () is the list of items recommended to the user . The equation 12 is normalized by the number of items. This equation is used to measure popularity bias on the item and provider sides.
The third criterion is Novelty, calculated as: 1 ∈
This Equation indicates that the novelty of the recommendation list decreases as the popularity of items increases, emphasizing a preference for less popular items. The study employs these criteria to compare and evaluate the results of diferent methods addressing the long-tail problem in recommender systems.
Equation 14 introduces a measurement for intra-user
diversity proposed by Zou et al. [
() =1 log2( + 1)
In this Equation, () is an indicator function that determines if item is relevant to user . A value of 1 indicates that item is relevant, while a 0 indicates that item is irrelevant.
NDCG provides a normalized measure of the efectiveness of a recommendation list by considering both relevance and the position of items within the list.
In this section, the study compares and analyzes the results obtained from various methods using the criteria introduced in the section above. For comparison, we use a real life scenario where the length of recommendation lists in all algorithms is considered to be 10.
The results in Table 1 indicate that MORS algorithms outperform baseline algorithms in addressing the long-tail problem. These algorithms demonstrate superior performance in diversifying items in recommendation lists, effectively increasing the participation of unpopular items. Notably, the study highlights that MORS algorithms achieve this diversification without compromising the accuracy of the recommendation lists. Therefore, the MORS algorithms are successful in preserving accuracy while simultaneously enhancing the inclusion of less popular items in the recommendations, addressing the long-tail problem in recommender systems.
The comparison table suggests that while MORS algorithms efectively mitigate popularity bias in recommendation lists, there is not a significant diference in the diversity of providers between baseline algorithms and MORS algorithms. For example, CF-User has a value of 0.5230 in AG-providers, while Malekzadeh and Wang show 0.5697 and 0.5711, respectively. Although MORS algorithms, aided by item diversifying objectives, ofer providers a better chance to present their items, the disparity in aggregate diversity is more noticeable on the item side than on the provider side when comparing MORS algorithms with baseline algorithms. Moreover, the comparison indicates that Baseline algorithms with higher accuracy than MORS algorithms exhibit poor performance in aggregate diversity, suggesting that recommendation list accuracy can negatively impact provider-side fairness. Specifically, CF-items achieve an accuracy of 0.7163, whereas CF-Users attain 0.6622. However, AG-providers exhibit respective values of 0.5067 and 0.5230.
Also, Table 1 indicates that among MORS algorithms, Malekzadeh’s work outperforms Shafiloo and Wang’s work in terms of the precision metric. However, this superiority adversely impacts aggregate diversity on both the provider and item sides. Specifically, Shafiloo’s work exhibits a precision of 0.7989, with aggregate provider diversity at 0.6059 and aggregate item diversity at 0.6930. In contrast, Malekzadeh’s work achieves a precision of 0.8338, but the aggregate provider diversity decreases to 0.5711, and the aggregate item diversity is 0.6651.
Furthermore, in Figure 1, we present the provider frequency using a bucketing technique. Specifically, in this ifgure, providers are assigned to a bucket based on the number of items belonging to that specific provider that are represented in all recommendation lists generated by the algorithm. For instance, a provider is placed in bucket one if only one item from all items associated with that provider is present in all recommendation lists.
This figure shows that baseline algorithms exhibit a weakness in recommending items from providers who lack popularity. This is illustrated in the initial buckets, where baseline algorithms struggle to include more items from less famous providers. Conversely, the first part of the buckets shows that MORS algorithms provide a more significant opportunity for less-known providers to showcase their items in the recommendation lists, ofering more visibility.
In conclusion, our study highlights the significance of MORS algorithms in addressing the issue of bias in recommender systems and promoting fairness for both items and providers. Our findings reveal that while baseline algorithms can negatively impact the provider’s fairness, MORS algorithms, particularly those leveraging evolutionary techniques and introducing less popular items to the initial population of their algorithms, can efectively mitigate popularity bias and enhance the provider’s fairness. This emphasizes the importance of considering provider-side fairness in the development of recommender systems, as it is often neglected in current research.
Overall, our research contributes to the growing body of work on fairness and bias in recommender systems and emphasizes the crucial role of MORS algorithms, particularly those employing evolutionary approaches, in mitigating bias and promoting fairness for both items and providers. Our study provides insights into how existing work objectives can influence provider fairness. It highlights the need for future research to delve deeper into this issue to provide a more comprehensive understanding of the problem. The efectiveness of MORS algorithms for providers could be further enhanced if a specific objective function were dedicated to mitigating provider bias. The absence of such an objective function might limit the algorithms’ ability to address biases related to the popularity of providers in the recommendation process.