In settings such as e-recruitment and online dating, recommendation involves distributing limited opportunities, calling for novel approaches to quantify and enforce fairness. We introduce inferiority, a novel (un)fairness measure quantifying a user's competitive disadvantage for their recommended items. Inferiority complements envy, a fairness notion measuring preference for others' recommendations. We combine inferiority and envy with utility, an accuracy-related measure of aggregated relevancy scores. Since these measures are non-diferentiable, we reformulate them using a probabilistic interpretation of recommender systems, yielding diferentiable versions. We combine these loss functions in a multi-objective optimization problem called FEIR (Fairness through Envy and Inferiority Reduction), applied as post-processing for standard recommender systems. Experiments on synthetic and real-world data demonstrate that our approach improves trade-ofs between inferiority, envy, and utility compared to naive recommendations and the baseline methods.
1. Introduction tems attracts increasing research attention, driven both by ethical and legal motivations. Here we focus on recommending items with limited availability, such as job recommendation, online dating, and education resource recommendation. The need for users to compete for recommended items distinguishes this recommendation setting from more standard ones such as e-commerce, or movie or music recommendation, where items have practically unlimited availability. handed. For example, a job seeker who applies for their recommended jobs could fail to get employed if these jobs were also recommended to better qualified rivals. This competition aspect brings specific challenges to evaluate and improve the fairness of recommendation strategies— challenges that have hitherto not been recognized.
To discuss this setting, it is useful to consider two possibly distinct kinds of afinity between a user and an item: an item’s utility for the user (i.e. the user’s preference), and a user’s suitability (i.e. competitiveness) for the item. In traditional recommender systems, only utility is relevant, as suitability is directly related to the competitive nature of the setting.
We consider two ways in which unfairness can arise RecSys in HR’23: The 3rd Workshop on Recommender Systems for Human Resources, in conjunction with the 17th ACM Conference on
As an illustration, consider the scenario of two users, 1 and 2, and three items, ○ , □ , and △. Let the utility scores be represented by the matrix: ○ 0.2 □ 0.6 ○ 0.1 □ 0.8 where the first row represents user
second row represents 2’s scores. Similarly, let the suitability scores, or the chances of a user getting the item, be represented by the matrix: ○ 0.3 □ 0.9
○ 0.3 □ 0.8
Examples: Recommending only △ to both users results in no envy, as the recommendations are equivalent and thus neither user prefers the other’s. However, there is high inferiority, as 1 is less suitable than 2, and thus less likely to obtain △
. Recommending ○ to 1 and □ to 2 results in high envy as 2’s recommendation has higher utility for 1 than their own recommendation, but no inferiority, as both users are recommended an item that is only recommended to themselves. Recommending ○ to both users results in neither envy, nor inferiority, but has low utility for both users. What is the best recommenda
︂] △△00..79 ,
︂] △△00..84 . they enter a competition for that item. Only one or a few will win and obtain the item, leaving the others empty- envy and low inferiority for everyone.
When multiple users are recommended the same item, items recommended to both A and B. We argue that a
The illustration above shows that both fairness notions are necessary: minimizing inferiority tends to result in less preferred jobs being recommended, which, if left uncontrolled, risks increasing envy. Moreover, there is also a trade-of between utility and both notions of fairness, particularly but not exclusively with inferiority.
Given the high stakes involved in many applications of this setting (with job recommendation as a notable example), there is an urgent need to adopt these notions of fairness in practical applications. While there is some work on the related notion of congestion and some limited work has been done on envy in recommender systems (see Sec. 4), we are unaware of any research directly addressing this need. This paper fills that gap, by formalizing these concepts as well as by proposing the FEIR (Fairness through Envy and Inferiority Reduction) method for post-processing the results of any other recommendation algorithm, yielding recommendations with low envy and low inferiority, while still maintaining high utility. Our specific contributions are: Let = (1, . . . , ) be users, = (1, . . . , ) be items. A recommender system recommends items to every user. The utility matrix is an × matrix, where each entry , ∈ (0, 1) represents the utility of item to user , so that each row ,: represents the utility function of evaluated on all the items. The suitability matrix is also an × matrix where each entry , ∈ (0, 1) represents the suitability (matching degree), between user and item .
gives us the item-wise utility, but in recommendation we need to measure the utility of a list of items.
Note that this list is a -sized multiset constructed from with repeated recommendations allowed, although in practice the chance of repetition is very slim when ≪ .
Another motivation of allowing repetition is for mathematical convenience as shown in 2.5.
Let be an × counting matrix where each entry , ∈ N0 is the number of occurrences of job in the recommendation for . Then each row ,: represents 1. We propose and formalize inferiority as a new individual fairness concept that is complementary to envy, when recommending items with limited availability. To facilitate minimizing these no- the recommendation list for such that ∑︀=1 , = tions, we also derive their expected values with for all . We omit the superscript if the context is clear. respect to a probabilistic interpretation of recommendation algorithms, resulting in diferentiable versions. (Sec. 2.1.)
Definition 2.1 (User utility). The utility of the recommendation for job seeker is a simple summation of the utility of each job to in ’s list: 2. Leveraging these diferentiable versions, we propose the FEIR algorithm, a model-agnostic postprocessing method of the output scores of any upstream recommendation algorithm for all useritem pairs. FEIR seeks a fairer score matrix by solving a multi-objective optimization problem with the goal of minimizing the expected envy and inferiority, and maximizing the expected utility. (Sec. 2.2.)
fairness measures and utility in extensive experiments both on synthetic and real data. We also demonstrate superiority of FEIR compared with the baseline methods. (Sec. 3.)
In this section, we first give quantifications of utility, envy and inferiority in the deterministic setting and the probabilistic setting (Sec. 2.1). Second, we formulate the problem of finding a good recommendation strategy as a multi-objective optimization problem solvable by minimizing a weighted sum of loss terms, leading to the FEIR method (Sec. 2.2).
(, , ,:) = ∑︁ , , .
=1
Envy measures the comparative utility from each individual’s perspective. It captures the idea that an individual may feel envy towards another if another person’s recommended items have higher utility to them, wrt. their own utility.1 Definition 2.2 (User envy). The envy from to * is:
(, * , , ) = ∑︁ , (* , − , ).
=1
Inferiority represents the disadvantage of one user to another when they compete for the same items, such as applying for the same jobs. It is measured based on the suitability between users and items, represented by the matrix . 1If repetition is not allowed, the formulation would use an indicator function representing whether item is recommended to user .
The major drawback of this formulation is that its probabilistic counterpart (the process of sampling without replacement) follows the hypergeometric distribution, which is computationally dificult.
Further investigation is left future work.
Definition 2.3 (User inferiority). The inferiority from to * is:
Inferiority captures the diference in suitability between user and * towards all the common recommended items, i.e., inferiority is only concerned with items recommended to both users. Note that, if any item occurred more than once, we only count it once, this is to consider the competition between them over the same item only once.
Definition 2.4. Utility, envy, and inferiority on the system level are simply the averages of the positive user-level measurements:
(, , ) = 1 ∑︁ (, , ),
=1 (, , ) = 1 (, , ) = 1
∑︁ 1≤ ̸=* ≤
∑︁ 1≤ ̸=* ≤ max (0, (, * , , )) , (, * , , ).
the random variables X, indicate the number of times item is included in ’s list. By our setting X,: follows a multinomial distribution with parameters and ,:.
In this context of probabilistic recommendation, the expected values of a user’s utility, envy, and inferiority are given by the following Proposition: Proposition 2.5 (Expected user utility, envy, and inferiority).
EX∼ [(, , X,:)] = ∑︁ , , , =1
EX∼ [(, * , , X)] = ∑︁(* , − , ), , =1 (4) (5) (6) (1)
EX∼ [ (, * , , X)]
= ∑︁ max(0, * , − , ) =1 · (1 − (1 − , ))(1 − (1 − * , )).
Proof outline. For utility and envy, this follows from the (2) fact that EX∼ [. ] = , (the factor stemming (3) from ∑︀
=1 = ), and from linearity of the expectation operator. For inferiority, this follows from linearity of the expectation operator, and from the fact that (1 − (1 − , ))(1− (1− * , )) is the probability that both , and * , are non-zero integers, and thus the probability that min(1, , * , ) is equal to 1.
The max(0, · ) in the definition of the envy ensures that only positive contributions are counted, to avoid a negative envy in one user to compensate a positive envy in another. (Individual user utilities and inferiorities are always positive.)
For a recommendation system for users over items , represented by the × matrix , the expected utility, 2.1.2. Probabilistic setting envy and inferiority of the -sized recommendation X on the system level is the average of the expected values The discontinous nature of recommender systems as rec- of all users: ommending multisets of items makes it practically impossible to utilize the utility, envy, and inferiority from EX∼ [(, , X)] = 1 ∑︀=1 EX∼ [(, , X,:)], Def. 2.4 in an optimization-based approach. We will thus (7) develop probabilistic alternatives that are diferentiable.
We consider a probabilistic recommendation setting EX∼ [(, , X)] = 1 ∑︀1≤ ̸=* ≤ max(0, in which the recommendation strategy is represented EX∼ [(, * , , X)]), by a user-item mapping function, : × → [0, 1], (8) trhecaotmasmsiegnndsataiopnroobfathbeiliittyemtotoeathcehuusseerr.-Titoemmopdaeilrthoef rtehce- EX∼ [ (, , X)] = 1 ∑︀1≤ ̸=* ≤ ommendations, we assume an independent multinomial EX∼ [ (, * , , X)]. (9) process for each user, where each user has a diferent -sided uneven dice, and to recommend ( ≪ ) items, 2.2. Optimization by minimizing we throw the dice times and take the outcome as the combined losses recommendation.
Then a recommender strategy can be represented as The FEIR algorithm minimizes a combined loss function an × matrix ∈ [0, 1]× where each entry , defined from the expected utility, inferiority, and envy of is the probability of recommending to . All users’ - the recommendation system. It uses a gradient descent sized recommendation can be written as a random matrix based method to optimize the scores of the resulting X where each row X,: is a random vector for where recommendation strategy, represented by the matrix ′, the items, leaving other losses calculated globally. User by solving: where ℓ( ′, ), ℓ ( ′, ), ℓ( ′, ), and ℓ( ′) = ∑︀=1(∑︀ =1 ′, −
1)2 are the expected envy (Eq. 8), expected inferiority (Eq. 9), negative expected utility (Eq. 7), and a penalty term for making each row of ′ a probaWe evaluate recommendation strategies based on the one-time deterministic recommendation obtained from bility distribution, respectively. Parameters 1, 2, 3, the probabilistic strategy. For diferent s, let be 4 are weights for each term. The penalty term ℓ( ′) can be omitted if the matrix ′ is renormalized after the binary matrix obtained from the recommendation strategy by setting the item indices with the highest each update (e.g., using row-wise softmax as the acti- values in each row to be 1 and the rest to be 0. vation function). An important benefit of
Normalized system-level utility and fairness.
The being model-agnostic: any model capable of scoring all system-level utility, envy and inferiority for top- recom(10) sampling takes a random subset of users at each training step and calculates the losses within this subset. Item sampling takes a random subset of items at each training step and calculates the losses between all user pairs with respect to only those items. User-item sampling samples from both the users and items at each training step.
2.4. Metrics ℓ( ′, , ) = 1ℓ( ′, ) + 2ℓ ( ′, ) + 3ℓ( ′, ) + 4ℓ( ′), user-item pairs can be post-processed by FEIR.
Notes on probabilistic and deterministic settings. The probabilistic setting is more general and mathematically convenient, but real recommendation systems typically recommend (deterministically) each user the items with the highest probabilities. Thus, our experiments train with the probabilistic but evaluate with the deterministic measures.
Notes on available afinity types in current systems. In our definitions, represents the suitability of users to items, and quantifies the utility of items to users. The diference between them can create tension between envy and inferiority, as seen when a job seeker prefers unsuitable jobs. However, real-world applications typically use a single afinity score provided by an existing recomThus, for practical reasons, in our experiments only one set of afinities is used to calculate both envy and inferiority, except for one synthetic dataset. In these cases, utilities and suitabilities align, but tension between envy and inferiority still arises due to individual diferences in scores. This is illustrated by a toy example where the scores for users 1 and 2 with respect to items are ︂[ ○ 0.1 □ 0.9 ○ 0.4 □ 0.6
︂] △△00..58 , and recommending □ to both of them results in no envy and high utility, but high inferiority from 2 to 1. 2.3. Scaling-up methods With large scale data, we propose the following approximation methods: inferiority loss mini-batching, user sampling, item sampling, user-item sampling. Mini-batching randomly splits the users into ⌊/⌋ batches and at each step calculates the inferiority from users within the current mini-batch to all the users with respect to all mender system, combining both suitability and utility. the following competition indicators. The mean rank mendation are defined by Eq. 1, 2 and 3, albeit = in the experimental data. We also calculate the overall fairness as
(, , , ) = (, , ) + (, , ).
Let denote the naive recommendation, then we have the normalized system-level top- recommendation metrics defined as
(, , ) (, , ) where (, , ) = 0).
∈
{, , } (no normalized envy since
Competition faced by each user. To address RQ2, we use of job seeker is calculated as: rank() :=
=1 1 ∑︁ |, |, where
, = {* |, = * , = 1, * , > , }, which measures the average rank of user among her competitors for the same recommended items. The mean suitability gap of user is calculated as: gap() =
1 ∑︁ , max(1, |, |) * ∈,
1 ∑︁ =1
(* , − , ), which measures the average diference in suitability scores between user and her likelier competitors for the same recommended items. By averaging these metrics over all users, we can obtain an overall evaluation of the competition.
Multiple solutions comparison. For methods that can generate multiple solutions representing diferent levels of trade-ofs, we plot the Pareto frontiers to visually compare sets of solutions. Additionally, we use the following numerical metrics: 1. HV (hypervolume) measures the amount of the objective space (relative to a reference point) that is dominated by the points on the frontier. 2. Fairness above utility threshold min(|): the minimum value of a fairness metric among all solutions with utility higher than , where could be inferiority, overall fairness, mean rank or mean suitability gap. This allows us to assess how well a solution performs in terms of fairness for a given level of utility.
We also use four real-world datasets, all obtained from the same upstream job recommendation model based on [5]. Zhilian: Scores for 2,781 users and 6,568 items, sampled from a public dataset provided by a Chinese online recruitment platform. CareerBuilder: Scores for 7,459 users and 11,020 items, obtained from a public dataset provided by CareerBuilder. VDAB small: Scores for 1,186 users and 8,921 items, a random sample from a private dataset provided by a labor agency in Belgium.
VDAB large: Scores for 10,369 users and 66,898 items, a random sample from the same source as VDAB small, but including more data. Results of this dataset are omitted due to space limitation.2 3.2. Baselines
Item-side fairness. Although our focus is user-side fair- Standard recommendation (Naive). Given the scores ness, we also investigated the item-side fairness by com- between all users and items, the common practice is to paring the Gini index of item exposure ([1, 2, 3, 4]) before recommend the items with the highest scores to each and after FEIR post-processing. user.
Randomization of top scored items (Shufle). Randomly 3. Experiments Sample items from items with the top- (≥ ) scores. Congestion alleviation method (CA). Naya et al. [6] proTo evaluate the efectiveness of FEIR, we conduct exper- poses a congestion alleviation method based on linear iments to answer the following research questions: optimization that aims to decreases the competition in the job market by using optimal transport. CA casts the RQ1. How does FEIR compare to the baseline methods problem of minimizing congestion into a linear program in improving the trade-ofs between envy, inferior- where the objective is to maximize the element-wise prodity and utility? uct of the original probability matrix and the solution RQ2. Does FEIR decrease the competition measure- matrix under the constraint of evenly distributing the ments defined from rivals compared to the baseline probability of recommending each item. methods? Modified Round-Robin procedure (RR). Modified based on [7], RR sets a threshold for suitability, randomly orders the users and then in each round, allocates one 3.1. Datasets item for each user at each round such that this item is the most preferred one for this user with suitability greater than . rounds would be run for top- recommendation.
Unlike the other methods, RR is applicable only when and are both available.
In our experiments, we use a variety of synthetic and real-world datasets to evaluate the performance of our proposed method. There are three types of synthetic datasets. Random synthetic data with distinct suitability and utility (SU50): Two 50 × 50 real-numbered matrices generated from a truncated normal distribution (0, 1) representing suitability scores and utility scores for 50 users and 50 items. Random synthetic data with one set of scores: Matrices generated from a truncated normal distribution (0, 1) with varying ratios of number of users and items to investigate the efect of varying these ratios. Structured synthetic data with one set of scores: Two 20 × 100 real-numbered matrices that simulate specific scenarios: Item groups (IG) and User groups (UG). The IG dataset represents the scenario where certain items have generally higher scores across all users, while the UG dataset represents the scenario where certain users have generally higher scores across all items. 3.3. Experiment setting For our method FEIR, we initialize the parameters by applying a row-wise softmax to the given scores and use gradient descent based methods to minimize the loss function defined in Eq. 10. We perform a coarse search to find an appropriate learning rate, and then use this value to train the model with diferent combinations of loss weights to achieve diferent trade-ofs between envy, inferiority and utility. For the CA baseline, diferent entropic relaxation terms are used to roughly controls the 2Due to the size of VDAB large, we experimented several methods for scaling up, including sampling and mini-batching. The results are included in our online supplementary materials. (a) Synthetic ̸= . (e) VDAB small = 5.
(f) VDAB small = 50.
(g) VDAB small: Inferiority vs utility = 50.
(h) VDAB large = 20. trade-ofs. For the synthetic datasets, we train and evaluate strategies for the top 10 recommendation. For the real-world datasets, we train and evaluate strategies for diferent s, ranging from 1 to 100. For the VDAB large dataset only a medium size = 20 is trained and evaluated due to time limitations.
We explore all scaling-up methods with the VDAB small dataset with = 100, find all methods perform similarly besides item sampling. Therefore, we apply one method to each real-world dataset for a full range of s: mini-batching to the VDAB small dataset, user sampling to the Zhilian and Careerbuilder datasets, useritem sampling to VDAB large. 3.4. Results
Our proposed method, FEIR, and the baseline methods were evaluated on synthetic and real-world datasets. The results indicate that both FEIR and CA can consistently improve fairness over the naive recommendation approach, while sacrificing some utility. By varying the hyperparameters for the methods, diferent trade-ofs between fairness and utility were achieved. To compare the results, we plotted each solution as a point on a graph with (un)fairness as the -coordinate and utility as the -coordinate, and drew the Pareto frontiers.
Synthetic datasets. FEIR is clearly the best (Fig. 1c, 1d), followed by CA, although the latter tends to cover a smaller solution region. RR scarifies too much utility for fairness (Fig. 1a). Shufle performs unstably.
Interestingly, a closer look at one of our solutions for
UG shows that FEIR can simultaneously decrease the inferiority for both user groups (Fig. 1d), which is desirable as it does not require sacrifices from one group to benefit the other.
When recommending items using the naive recommendation strategy with the random synthetic datasets with varying user-item ratios, the inferiority increases with an increased ratio of users to items, indicating that the naive approach causes competitive disadvantages for users, and the more limitation the tenser the competition. CA does not decrease inferiority well when the number of items is not greater than the number of users; on the other hand, FEIR is able to find solutions with low inferiority as seen in Fig. 1b. When the number of items surpasses users, CA can also find solutions with low inferiority and high utility, but is still outperformed by FEIR (corresponding figures included in our online supplementary.).
Real world datasets. Data exploration confirms the existence of inferiority and competition caused by the naive recommendation. With increasing s, the utility per recommendation decreases, and the inferiority and competition increase with a decelerating growth rate (see ifgures in our online supplementary). The reason is that with a larger , there are more overlapping recommendation and more competition, but also the average scores decrease with increasing .
The VDAB small and CareerBuilder datasets show similar patterns in the relative performance of FEIR and CA.
FEIR can decrease inferiority without reducing much utility or increasing envy, while CA decreases inferiority but also increases envy and reduces utility, especially (a) Zhilian all users = 50.
(b) Zhilian one user cluser.
(c) IG: rank. (e) UG.
(f) VDAB small = 5.
(g) VDAB small = 50.
(h) Zhilian = 50. In general, CA is capable of achieving a low mean rank
FEIR CA FEIR CA FEIR CA FEIR CA (Fig. 2c), but always a much higher mean gap compared 1 0.043 0.013 0.048 0.031 0.140 0.642 0.006 0.006 to FEIR (Fig. 2d, 2e). We argue that FEIR is more de5 0.042 0.024 0.045 0.031 0.138 0.365 0.049 0.127 sirable. A recommendation with a low mean rank but a 2100 00..003491 00..002276 00..004424 00..003312 00..118453 00..332211 00..100841 00..117452 large mean suitability gap suggests that, although a user 50 0.034 0.026 0.035 0.029 0.278 0.357 0.248 0.248 does not have many competitors, the competitors she 100 0.029 0.025 0.030 0.027 0.367 0.392 0.333 0.324 does have are much better hence much more likely to defeat this user. For example, consider a job seeker with a suitability score of 0.7 for a certain job. CA tends when the number of recommendations is small. Shuf- to recommend this jobs to only one other job seeker with lfe prioritize utility, but cannot reduce much unfairness a score of 0.99, and on the other hand FEIR may rec(Fig. 1e and 1f). ommend this job to three other job seekers with scoring
With large = 50, FEIR’s performance in reducing 0.69, 0.74, 0.8 respectively. It is reasonable to believe unfairness is not as good as CA for the Zhilian dataset that FEIR gives user a better chance of getting hired, when user sampling is used (Fig. 2a). Nonetheless, FEIR especially when considering that in reality, one would performs better than CA when trained on smaller subsets not apply for all recommended jobs. Shufle performs of users that can be processed in a single batch as seen in almost always the worst.
Fig. 2b. This suggests that the loss functions are efective, When recommending a small number of jobs from a but the decreased performance is most likely due to the large pool, CA sometimes recommends non-overlapping optimization process or some unique characteristics of jobs to each user, resulting in trivial solutions with no the Zhilian dataset, which is left for future work. competition but decreased utility, as seen in the left most
FEIR performed well on the VDAB large dataset, even region of Fig. 2f. However, FEIR can provide solutions with a sample size relatively small to the total numbers with higher utility. As increases, it becomes harder to as show in Fig. 1h. give non-overlapping recommendations for CA such that
Quantitative comparisons of the Pareto frontiers gen- FEIR always gives a lower suitability gap (Fig. 2g, 2h). erated by FEIR and CA for the VDAB small, Zhilian and A quantitative comparison of the Pareto frontiers gen
Competition and congestion in recommendation. To the
best of our knowledge, there has been no research at all
on the concept of inferiority. Yet, Naya et al. [6] did study
the related notion of congestion, in the context of labor
market. They proposed a congestion alleviation method,
which reduces the intersection between the sets of jobs
recommended to diferent job seekers. Congestion does
erated by FEIR and CA for the VDAB small dataset with not consider suitability (i.e. competitiveness) of users for
various values shows that FEIR is better than CA al- their recommended jobs like inferiority does.
most across the board, except for min(rank|0.9) with Envy-freeness in recommendation. Inspired by the
lit = 50 and 100 (Table 2). The CareerBuilder dataset has erature on social choice theory and fair resource
allocasimilar results with VDAB small where FEIR is better tion (e.g., [
Our code and supplementary materials for more details the item side while guaranteeing envy-freeness up to
and extra plots are publicly available at https://github. one good (EF1) fairness for every user, and Wu et al. [
This paper extends the growing literature on fairness in machine learning (e.g. [8, 9, 10, 11, 12, 13, 14, 15, 16, 7]).
Here we summarize the most directly related research.
Fairness when recommending items with limited
availability. Particularly in the context of job
recommendations, this is an increasingly active research area. Yet, the
current literature mainly focuses on group level disparity
notions. For example, Geyik et al. [
Recommending items with limited availability to users has its own challenges and brings new fairness requirements not addressed in the existing literature. In this paper we proposed envy and inferiority as important fairness notions to fill the gap and presented a postprocessing approach FEIR to improve the fairness of such recommendation settings.
Our experiments on synthetic and real job recommendation datasets demonstrated that FEIR improves fairness by reducing the potential competitive disadvantage of users without significantly sacrificing utility. Importantly, our method FEIR is not limited to the labor market, but also promising in reducing user inferiority and com
The research leading to these results has received funding from the European Research Council under the European Union’s Seventh Framework Programme (FP7/2007-2013) (ERC Grant Agreement no. 615517), and under the European Union’s Horizon 2020 research and innovation programme (ERC Grant Agreement no. 963924), from the Flemish Government under the “Onderzoeksprogramma Artificiële Intelligentie (AI) Vlaanderen” programme, and from the FWO (project no. G0F9816N, 3G042220). Part of the experiments were conducted on pseudonimized HR data generously provided by VDAB (Vlaamse Dienst voor Arbeidsbemiddeling en Beroepsopleiding). petitive disadvantages in other real-world scenarios such as online dating, paper bidding systems, and education resources recommendation.
Our work has limitations but also opens up new research opportunities. The actual competition and chances of getting any item depend on many factors beyond any recommendation system and hence beyond our scope. Also, emphasizing envy and inferiority does not make other existing fairness concerns any less important, nor the case that they can cover all new fairness requirements from the unique features of recommending limited resources. Rather, our findings create new opportunities for research to explore the relations among diferent fairness notions and identify other ignored dimensions of fairness in these settings.
Some alternative formulations of utility, envy and inferiority are possible. For example, disallowing repeated recommendation for a user, which involves further complexity in the probabilistic setting. It is also possible to modify the quantification of inferiority by taking the utility into account. The analysis and comparison of the current formulation and the alternatives would be interesting for future work. Besides, the interests of recruiters could be further considered by adapting the optimization objective to include some metrics representing the suitability of candidates. The dynamics between job seeker side and recruiter side fairness is another future direction worth exploring.