Efforts to reduce social bias in machine learning has increased in the past several years. As data privacy concerns grow, finding techniques to train private, debiased machine learning models becomes increasingly important. Federated Learning (FL) has emerged as a popular privacy-preserving machine learning strategy. FL, however, by not providing complete access to training data, brings with it a unique set of difficulties in bias mitigation that have yet to be explored. In this paper, we delve into these difficulties, and how they can affect bias measured in federated learning models.
Machine learning applications are used throughout the world
to make decisions on matters with real-world consequences
[
Many bias mitigation efforts focus on centralized machine
learning [
FL maintains data privacy by allowing multiple parties to
collaboratively train a model without sharing their data. FL
is already utilized in real-world settings, i.e. Google Gboard
with cell phones, enterprise frameworks such as IBM
Federated Learning [
CCooppyyrigghhtt©©20220121fo,r Athsisopcaipaetirobnyfiotsr atuhtehoArsd.vUasnecepmeremnitteodf uAnrdtiefircial ICntreelaltiigveenCcoem(wmwonws.Laiacaein.osregA).ttAribllutriiognh4ts.0reInsteerrvneadti.onal (CC BY 4.0).
As seen in Figure 1, during each round of training, parties (Pn) train a local model (M1), and are queried (Q) by the aggregator (A) to send model updates (Rn); these updates can be model parameters or a party’s local model. The aggregator utilizes these to update the global model, usually following a specific model fusion strategy. This strategy, called the fusion algorithm, takes the model updates (R1...Rn) as input and can combine them in a myriad of ways, based on the design. It may simply average each party’s weights, or perform a more involved aggregation, then return the global model. This process repeats for several rounds until a certain accuracy or maximum number of training rounds has been reached.
Designed for centralized machine learning, most bias mitigation approaches measure and reduce undesired bias with respect to a sensitive attribute, such as race or sex, in the training dataset. Unfortunately, existing techniques are not directly applicable to the federated learning setting. As federated learning bars full access to the training dataset, finding methods to address bias without directly examining sensitive information is an open challenge. FL is unique, and as the number bias mitigation methods in centralized machine learning grow, it is increasingly apparent that there are additional considerations to be made about how bias affects federated learning models.
In this paper, we investigate four sources of bias, three unique to training in the federated learning setting. These are traditional bias sources, party selection and subsampling, data heterogeneity, and fusion algorithms. We hope this discussion will stimulate research in this nacient field.
Each party in a federated learning process trains their local
model similarly to how it would be trained in a centralized
machine learning. Factors, such as prejudice,
underestimation, and negative legacy [
However, new sources of bias occur when training in a FL fashion. Through the model updates shared at each round, each party will introduce its own biases to the global model.
In the following, we introduce unique challenges to federated learning algorithms, setting and processing.
Unlike in centralized machine learning, federated learning faces a unique challenge related to data heterogeneity. Parties each have a local dataset that they use to train their local model. It is very possible that each party’s subgroup of data differs greatly from the overall data composition from all parties involved. Additionally, in FL parties might be capable of dynamic participation, where they can drop out of a FL process and rejoin later on for various reasons, i.e. connectivity restraints. Overall and relative data composition thereby may be constantly changing, which affects how the global model learns bias.
For example, say a chain of hospitals wants to use federated learning to train an image classifier for detecting heart disease. Each hospital, in a different location, trains their local model with their patients’ data. A hospital in a predominantly minority neighborhood of a larger, predominantly non-minority city is likely to have a very different set of patients in its local dataset, relative to the overall composition of the hospital chain’s set of patients.
When an aggregator incorporates local model updates into
the global federated model, it follows a strategy dictated
by a fusion algorithm. Fusion algorithm designs vary, and
influence the bias measured in the final model. For
example, some are designed to equally incorporate model updates
from parties, while others may perform a weighted average
based on party size (i.e. parties with larger datasets influence
the global model more than parties with smaller datasets)
[
In the same hospital example as above, some hospitals training together have very different dataset sizes. This could be based on some hospitals being located in areas where the population is socioeconomically disadvantaged, thereby those hospitals have less patients that can afford an expensive medical procedure; these hospitals’ datasets would be smaller. Involvement in a federated learning process that rewards larger party size with more global model influence would diminish these hospitals’ contributions to the global model, and incorrectly give the impression that the model is comprehensively learning from the hospital chain’s set of patients. This example can be easily reproduced for situations where other sensitive attributes like age, sex, or race can affect whether or not a user’s data is systemically kept out of a federated learning task.
Parties involved in a federated learning process are queried
by the aggregator at each round of training to send their
model updates, which the aggregator will then incorporate
into the global federated model. However, not all parties
might be involved in every round of training [
Consider a scenario where a company wants to train a model to improve the user experience in its cell phone app, and engages users in a FL process. In this example, each phone is a party, and the question of which model updates are included is dependent on network speed. Faster devices, which are likely to be newer, are likely to be represented at disproportionately higher rates than slower devices. Likewise, devices in geographic regions with slower networks may be represented at disproportionately lower rates. Inclusion here is correlated with socioeconomic status, and is a systemic source of bias.
Bias identification and mitigation in centralized machine learning has been discussed in recent years. Since then, federated learning has emerged and been rapidly put into practice, due to its potential to mitigate privacy and regulatory concerns related to machine learning. The shift from centralized to federated learning comes with challenges that stem from limited access to training data, alongside other barriers. Overcoming these obstacles is key to understanding new sources and causes of bias in FL settings. We have identified four causes of bias affecting federated learning, three of which are unique to the FL process. Successful solutions to mitigate bias should consider multiple components of the FL process, and ultimately ensure data privacy is maintained. We hope this paper inspires further exploration in this area.