Training machine learning (ML) models for natural language processing usually requires large amount of data, often acquired through crowdsourcing. The way this data is collected and aggregated can have an e ect on the outputs of the trained model such as ignoring the labels which di er from the majority. In this paper we investigate how label aggregation can bias the ML results towards certain data samples and propose a methodology to highlight and mitigate this bias. Although our work is applicable to any kind of label aggregation for data subject to multiple interpretations, we focus on the e ects of the bias introduced by majority voting on toxicity prediction over sentences. Our preliminary results point out that we can mitigate the majority-bias and get increased prediction accuracy for the minority opinions if we take into account the di erent labels from annotators when training adapted models, rather than rely on the aggregated labels.
When using crowdsourcing to gather training data for Machine Learning (ML) algorithms, several workers work with the same input samples and the annotations are aggregated into a unique one like the majority vote (MV) to ensure its correctness (elimination of annotation mistakes and spammers mainly). Although this data collection method is designed to get high-quality data, we expect that certain tasks involving subjectivity such as image aesthetic prediction, hate speech detection, detection of violent video segments, sentence sentiment analysis, cannot be tackled this way: samples should not be described with unique labels only since they are interpretable di erently by di erent persons.
The use of hate/toxic speech has increased with the growth of the
Internet [
We show on the toxicity dataset [
Subjectivities in the dataset. For each worker, we compute the
average disagreement rate (ADR) with the ground truth (percentage of annotations
di erent from the MV here), and plot the distribution over the dataset after
removing the annotations of the lowest quality workers (spammers) ( g. 1). The
quality score for each worker (WQS) is computed with the CrowdTruth
framework [
Algorithmic e ect of the bias. We consider the task of predicting binary labels. Training traditional algorithms to predict the MV, annotations of only maximum 0.09% of annotators would be entirely correct: the majority-bias is not consistent with the worker's individual opinions. We evaluate traditional models (sec. 3) trained and tested on aggregated and disaggregated labels (table 1). In both cases accuracy is higher when measured on aggregated data, what shows that classical input data's treatment makes usual models' predictions biased towards one type of opinion, here the majority opinion, instead of representing each subjectivity.
We claim that a fairer algorithm should return di erent outputs for a same sample depending on its reader. Here, we propose measures of the majoritybias' algorithmic e ect and a method to counter its unfairness.
Bias measure. Global metrics are usually used to optimize the algorithms' parameters and evaluate them. However, they do not inform on the bias' effects since most samples' labels have a high-agreement: the slight improvement when training on disaggregated data hints only lightly at label disaggregation (table 1, g. 2). To identify the e ects, we propose to measure sentence-level and worker-level accuracies on the annotations spread in the following bins: we divide the sentences along their ambiguity score (AS) (percentage of agreement in annotations) or UQS, the workers with their ADR, WQS or demographics categories; and also plot histograms of the per-user and per-sentence errors to identify potential unfairness among all workers or sentences.
Bias mitigation: ML. To account for the full range of valid opinions, we
propose to modify the inputs to the ML models. After removing low-quality
workers, instead of the aggregated labels we feed them with the annotations
augmented with the available worker demographics (age, gender, education, with
a continuous or one-hot encoded representation) that psychology literature [
Bias mitigation: dataset balancing. We de ne 4 data set-ups to help the algorithms learn the individual annotations. Sentence AS and MV-toxicity are computed, and we resample the dataset following the original distribution or balancing the distribution on these 2 criteria, to obtain a dataset whose majoritybias is decreased by equally representing samples with high and low agreement between workers. We also resample the annotations along the MV-toxicity and demographics categories (removing the least frequent ones) into one dataset following the distributions and a balanced one, to foster performance fairness in-between populations.
Results. Binned metrics like the user-level ADR-binned accuracy ( g. 2 with bins along the y-axis) enable to show that models are more suited to workers who agree with the MV (bottom of the y-axis), and highlight the bene t of using disaggregated data with adapted ML models. On the AS-balanced dataset (left part of the x-axis), the user representation increases accuracy for workers with a high disagreement with the majority over using aggregated data or no usermodel. The resampling choice also helps understanding and mitigating bias' e ects: balancing on demographics neither clearly shows the performance gap between minority and high-ADR workers nor improves accuracy with the user representation, contrary to the AS dataset in which MV-consensus' presence is reduced. 4
Disaggregating the annotations decreases the majority-bias' e ects with adapted ML models' inputs and dataset resamplings. Binning the evaluation metrics enables to understand and verify the existence of these e ects. We only reported results using the LR classi er but we now investigate adaptations of Deep Learning algorithm's architectures which are better suited to the large dataset (10 times more annotations than labels) and to the size of the ML inputs.
This research is supported by the Capture Bias project 3, part of the VWData Research Programme funded by the Startimpuls programme of the Dutch National Research Agenda, route "Value Creation through Responsible Access to and use of Big Data" (NWO 400.17.605/4174).