Fair algorithms are a prerequisite if we want to increase user trust, but are meanwhile also required for legal reasons. Increasing the fairness of a model can lead to a decrease in prediction accuracy. While recent work on fairness is often based on developing models that optimize fairness metrics while keeping the reduction in accuracy to a minimum, few focus on how to make this fairness-performance trade-of as controllable as possible. Several methods include a parameter for specifying the level of fairness penalty, leading not necessarily to a smooth trade-of curve. By this we mean a curve that deviates only slightly from being monotonically non-increasing. In this paper, we show that by only giving the sample weights generated by an AdaBoost-based variant that is predicting the sensitive attribute to another classifier, discriminative predictions can be reduced and the trade-of can even be made more controllable. Thus, our method allows for the possibility of creating a more strategic balance between fairness and performance of the model through user control.
= ︂( 1 − ( ) · )︂ 2 · , (1) where corresponds to the current iteration of AdaBoost. We hand over the sample weights of all iterations of ℎ as well as the model weights of all weak learners to the subsequent ensemble . Following this, the training process of consists of fitting model , = 1, ..., on and assigning weights 1 − to its ensemble members.
For evaluation, we perform a 15-times hold-out in the outer loop, splitting the data randomly into 23 of training data and 13 of test data each time. Selected hyperparameters are then optimized on the training data using three-fold cross validation in an inner loop. We optimize all models on the area
CEUR
ceur-ws.org 0.65 C RO 0.6 U A 0.55 0.5 0.65 C RO 0.6 U A 0.55
0.8 0.85 0.9 0.95 1 - Statistical parity diference
1 (a) FairTrader 0.75 0.8 0.85 0.9 0.95 1 - Statistical parity diference
1 (b) Adversarial mitigation
80 60 40 20 0 80 60 40 20 0
0.6
C
O
RU0.5
A
under the trade-of curve (AUTOC), as points should ideally be close to the top-right corner in order
to increase performance as well as fairness measures. For this purpose, the curves are truncated at
their minimum and maximum -values, which also determines the extent to which a curve explores
the trade-of space. We compare to the adversarial mitigation method of Zhang et al. [
Taking the best of all models in terms of AUTOC leads to the preliminary results illustrated in Table 1 and Figure 1. We notice that for the best chosen model for the Compas dataset, the points of the adversarial mitigation method lie for the most part slightly above those of our method. However, we do not consider only the points indicating one specific pair of numbers of the trade-of, but take also a look at the progression of the points in the order they were measured when increasing the fairness parameter, here illustrated by colors. We observe that for adversarial mitigation and the reduction approach, it is less clear to estimate where the next calculated pair of fairness and performance measure will actually lie if we run the algorithm with gradually increasing , even though represents the balance between training towards predicting y and enforcing the fairness constraint. On the other hand, for FairTrader, we can say that the trade-of curve iteratively decreases monotonically in the majority of cases, especially compared to adversarial learning.