The increasing use of Artificial intelligence (AI) in the medical domain has highlighted a critical issue: bias in datasets. Biases in medical datasets can lead to skewed predictions, unfair clinical decisions, incorrect diagnoses and poor generalisation of AI models. Very often, these biases are the consequence of imbalance in the dataset. Generative Adversarial Networks (GANs) have appeared to be a promising solution for solving the data imbalance issue. Synthetic data can help mitigate bias by balancing the dataset for sensitive attributes as well as for class labels. However, the eficiency of diferent GAN variants in mitigating bias remains unexplored in the medical domain. This paper investigates and compares various GAN variants to identify the most efective approach to producing balanced data. In this study, we evaluated diferent variants of GAN on three medical datasets with the aim of contributing to the development of more fairer and inclusive AI models in the medical domain. The study shows that the performance of the Machine Learning (ML) model improves when the dataset is balanced using synthetic data samples. Moreover, the MedGAN variant performs better when compared with other variants of GAN.
community [13].
GAN is a deep learning model that mainly consists of two neural networks: a Generator used to generate artificial data and a discriminator that tries to distinguish between real and synthetic data to improve quality. These models were first introduced to process only image data, but later diferent variants of GAN were proposed to process tabular data as well. These variants include Tabular GAN (TGAN) [14], Conditional Tabular GAN (CTGAN) [15], Medical GAN (MEDGAN) [16], Multi-Categorical GAN (MC-MedGAN) [17] and many more.
In this study, we evaluated various GAN variants including GAN, TGAN, CTGAN, MedGAN, and MC-MedGAN to generate synthetic samples to balance diferent group representations within medical datasets. The newly balanced dataset was fed into diferent ML models including Logistic Regression (LR), Random Forest (RF), Decision Tree (DT), and K-Nearest Neighbour (KNN) to draw a comparison. The GAN models are evaluated on three diferent medical datasets that consist of gender as a sensitive attribute to balance: the Asthma Disease Dataset [18], the Heart Disease Prediction Dataset [19], and the Cancer Prediction dataset [20]. The performance is evaluated using various metrics i.e., accuracy, precision, F1-score, recall, and Area Under Curve (AUC) scores. Fairness is evaluated using Equal Opportunity (EO) [21], Propensity Score (PS) [22], and Statistical Parity (SP) [23].
In today’s world, AI is an integral part of the healthcare system. The AI model must incorporate
transparency and accountability. The goal of this research is to reduce bias in medical datasets that
contain inherent biases due to unequal representation of diferent demographic groups. AI models
can become unfair and imbalanced, particularly in the healthcare sector, where underrepresented
groups may receive scant care. Bias in medical datasets poses a significant challenge to the reliability
of predictive models [24]. This could be critical for healthcare systems since an automated model
prediction has a direct efect on patients that afects their mental health, and quality of life or may
risk the life of an individual [25] as well it also leads to financial loss [
Hypothesis: GAN-based data generation methods can help to reduce biases and ensure fairness in medical datasets.
The formulated research questions to explore the above hypothesis are as follows: • Does GAN-based synthetic data generation help reduce biases in medical datasets? If yes, which
GAN variant performs better among basic GAN, TGAN, CTGAN, MedGAN, and MC-MedGAN? The rest of the article is structured as follows: Section 3 highlights some of the recent related work. Section 4 explains the methodology in detail. Section 5 explains the results. Section 6 discusses the hypothesis and research questions and Section 7 concludes the discussion with future work.
GANs have gained significant attention in recent years due to their capability of generating
highquality data. Therefore, this section reviews the recent methodologies that leverage GAN models to
generate synthetic data. A study [
In another study [
The above studies show that GANs have the potential to generate high-quality diverse datasets that can be used to handle bias in real-world datasets. Therefore, to analyse the capabilities of diferent GAN variants, this study aims to conduct multiple experiments and then assess the fairness within the newly generated synthetic medical datasets.
The data preprocessing includes one hot encoding to replace categorical variables with numerical
numbers. Afterwards, we applied z-score normalisation on each distinct numerical feature because they
did not contain extreme outliers [
In order to balance the dataset for the sensitive attribute i.e., Gender and class labels. We employed
ifve GAN architectures: basic GAN, TGAN, CTGAN, MedGAN and MC-MedGAN. These variants are
specifically designed to handle tabular and medical datasets which is the primary focus of our study.
GAN is a type of neural network architecture where two networks, a generator, and a discriminator,
are trained simultaneously [
After generating synthetic samples to balance the datasets for sensitive attribute (gender) and class labels, diferent commonly used ML classifiers including Logistic regression (LR), Random Forest (RF), Decision Tree (DT), K-Nearest Neighbour (KNN) with default parameters are trained on the newly generated datasets to evaluate the performance of GAN variants.
To evaluate the performance of GAN variants, we used three diferent medical datasets that contain sensitive attributes. The details of each of these datasets are as follows:
Asthma Disease Dataset: The Asthma Disease Dataset [18] contains a record of 2,392 samples with 28 features. The output label is the diagnosis indicator, which is taken as 0 for the absence and 1 for a positive case. It contains 2,268 samples for class 0 as compared to 124 samples with class label 1. Also, the number of samples for males is 1212 whereas for females the count is 1180.
Heart Disease Prediction Dataset: The Heart Disease Prediction Dataset [19] consists of 13 features and 303 samples. The dataset contains 207 male and 96 female samples.
Cancer Prediction Dataset: The Cancer Prediction Dataset [20] contains 1,500 samples with 8 features. The target variable ’diagnosis’ indicates whether a patient has cancer or not (0 for no cancer and 1 for cancer). The diagnosis distribution shows 943 patients without cancer and 557 with cancer. There are 736 female samples and 764 males in total.
The performance is evaluated by training diferent ML classifiers as mentioned in Section 4. The classifiers are assessed using accuracy, f1-score, precision, recall and AUC. Whereas the fairness of the dataset is evaluated via EO, PS, and SP. EO guarantees that all individuals receive the same treatment and meet the same requirements [21]. PS can be defined as the conditional probability of being exposed to a treatment given the observed covariates [22]. SP is a fairness criterion that requires the probability of a favourable outcome to be the same for each demographic group [23]. Tables 1, Table 2, and Table 3 show each classifier’s performance on the original as well as on each generated dataset. It can be seen that MedGAN performs well for the Asthma Disease Dataset and Cancer Prediction Dataset while MC-MedGAN has a better score for the Heart Disease Dataset.
Figure 2, shows the fairness metric performance on the Asthma Disease dataset. The SP, PS, and EO scores improve when the dataset is balanced for class label and gender. MEDGAN has a better performance for all three datasets followed by MC-MedGAN and TGAN. The same performance is observed for the other two datasets. The other graphs are given in Appendix A.
Overall, the results show that balancing the dataset for class labels and sensitive attributes improves the performance as well as the fairness of the model. Among diferent GAN variants, the MEDGAN produces good results and lower statistical, propensity and equal opportunity scores showing its great capability for reducing bias followed by MC-MedGAN. Moreover, the predictive ability of RF classifiers is better than other classifiers in terms of accuracy, precision, recall, f1-score, and AUC.
Method
Model LR RF DT KNN LR RF DT KNN LR RF DT KNN LR RF DT KNN LR RF DT KNN LR RF DT KNN Model LR RF DT KNN LR RF DT KNN LR RF DT KNN LR RF DT KNN LR RF DT KNN LR RF DT KNN (a) (b) Opportunity (c) Figure 2: Fairness Assessment for Asthma Disease dataset (a) Statistical Parity (b) Propensity Score, (c) Equal
This section discusses the overall findings of the study in view of the literature review and extensive experimentation conducted to analyse our hypothesis. Based on our research question, the experiments show that classifier performance as well as the fairness metrics score improves when the datasets are balanced for sensitive attributes and class labels. Figure 2 shows the improvement in the fairness scores across each metric when the dataset is balanced via synthetic data generation using GAN variants as compared to the original dataset. Moreover, the analysis of each GAN variant based on performance evaluation using accuracy, precision, F1-score, recall, AUC and fairness metrics via EO, PS, and SP indicates that the MedGAN produces eficient performance followed by MC-MEDGAN across all three datasets. To validate any statistically significant diference between these two methods, we applied a paired t-test on the EO, PS, and SP scores for each of these methods. The p-values for EO, PS, and SP came out to be 0.34, 0.61, and 0.30 respectively. Therefore, we fail to reject our hypothesis and conclude that these two methods are not significantly diferent. These GAN variants are specifically designed for medical datasets to capture the interdependencies between the diferent variables to generate synthetic data similar to original data properties [16, 17]. However, further experimentation with other datasets including post-hoc tests will be conducted in future to provide deeper insights into the capability of GAN variants for data generation.
In this paper, we tested diferent types of GANs for their capacity to produce synthetic tabular data to decrease bias in medical datasets. Our key findings are GAN-based models are efective for bias migration and GAN can provide a balanced dataset to produce generalised AI models and provide a solution AI for all and AI for good. On the other hand, traditional GANs were successful but medical domain-based GANs displayed greater performance in generating high-quality and unbiased data. It drives us to have more specific models in the future. Despite certain advantages of the GAN, we face some obstacles such as evaluation metrics. There is a need to have more standardised and compressive evaluation metrics of this model focused on decreasing bias. The studies in this article suggest that synthetic data can assist in eliminating bias and improve the efectiveness of the classifier. Moreover, MedGAN performs better in terms of SP, PS, and EO. In future, we will extend our work for various variations of GAN focused on refining GAN architecture to adapt the multimodality medical data, bias-sensitive evaluation mechanism and testing the GAN-based techniques in real-world clinical data.
This research was managed by the CREATE-DkIT project, supported by the HEA’s TU-Rise programme and co-financed by the Government of Ireland and the European Union through the ERDF Southern, Eastern Midland Regional Programme 2021-27 and the Northern Western Regional Programme 2021-27. This research is also partially supported by the Research Ireland under Grant Number 21/FFP-A/9255. [4] H. A. Haenssle, C. Fink, R. Schneiderbauer, F. Toberer, T. Buhl, A. Blum, A. Kalloo, A. B. H.
Hassen, L. Thomas, A. Enk, et al., Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists, Annals of oncology 29 (2018) 1836–1842. [5] H. Suresh, J. Guttag, A framework for understanding sources of harm throughout the machine learning life cycle, in: Proceedings of the 1st ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization, 2021, pp. 1–9. [6] R. González-Sendino, E. Serrano, J. Bajo, P. Novais, A review of bias and fairness in artificial intelligence (2023). [7] S. E. Eskimez, D. Dimitriadis, R. Gmyr, K. Kumanati, Gan-based data generation for speech emotion recognition., in: INTERSPEECH, 2020, pp. 3446–3450. [8] M. A. Shahul Hameed, A. M. Qureshi, A. Kaushik, Bias mitigation via synthetic data generation: A review., Electronics (2079-9292) 13 (2024). [9] N. V. Chawla, K. W. Bowyer, L. O. Hall, W. P. Kegelmeyer, Smote: synthetic minority over-sampling technique, Journal of artificial intelligence research 16 (2002) 321–357. [10] J. Chakraborty, S. Majumder, T. Menzies, Bias in machine learning software: Why? how? what to do?, in: Proceedings of the 29th ACM joint meeting on european software engineering conference and symposium on the foundations of software engineering, 2021, pp. 429–440. [11] H. Han, W.-Y. Wang, B.-H. Mao, Borderline-smote: a new over-sampling method in imbalanced data sets learning, in: International conference on intelligent computing, Springer, 2005, pp. 878–887. [12] T. Jo, N. Japkowicz, Class imbalances versus small disjuncts, ACM Sigkdd Explorations Newsletter 6 (2004) 40–49. [13] A. Figueira, B. Vaz, Survey on synthetic data generation, evaluation methods and gans, Mathematics 10 (2022) 2733. [14] L. Xu, K. Veeramachaneni, Synthesizing tabular data using generative adversarial networks, arXiv preprint arXiv:1811.11264 (2018). [15] L. Xu, M. Skoularidou, A. Cuesta-Infante, K. Veeramachaneni, Modeling tabular data using conditional gan, Advances in neural information processing systems 32 (2019). [16] E. Choi, S. Biswal, B. Malin, J. Duke, W. F. Stewart, J. Sun, Generating multi-label discrete patient records using generative adversarial networks, in: Machine learning for healthcare conference, PMLR, 2017, pp. 286–305. [17] A. Goncalves, P. Ray, B. Soper, J. Stevens, L. Coyle, A. P. Sales, Generation and evaluation of synthetic patient data, BMC medical research methodology 20 (2020) 1–40. [18] R. E. Kharoua, Asthma disease dataset, 2024. URL: https://www.kaggle.com/datasets/ rabieelkharoua/asthma-disease-dataset, accessed: 2024-08-23. [19] K. Ujeniya, Heart disease prediction dataset, 2024. URL: https://www.kaggle.com/datasets/ krishujeniya/heart-diseae/data, accessed: 2024-08-23. [20] R. E. Kharoua, Cancer prediction dataset, 2024. URL: https://www.kaggle.com/datasets/ rabieelkharoua/cancer-prediction-dataset/data, accessed: 2024-08-23. [21] M. Hardt, E. Price, N. Srebro, Equality of opportunity in supervised learning, Advances in neural information processing systems 29 (2016). [22] A. E. Valojerdi, L. Janani, A brief guide to propensity score analysis, Medical journal of the Islamic
Republic of Iran 32 (2018) 122. [23] A. N. Carey, X. Wu, The statistical fairness field guide: perspectives from social and formal sciences, AI and Ethics 3 (2023) 1–23. [24] J. Gesi, X. Shen, Y. Geng, Q. Chen, I. Ahmed, Leveraging feature bias for scalable misprediction explanation of machine learning models, in: 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE), IEEE, 2023, pp. 1559–1570. [25] J. W. Gichoya, K. Thomas, L. A. Celi, N. Safdar, I. Banerjee, J. D. Banja, L. Seyyed-Kalantari, H. Trivedi, S. Purkayastha, Ai pitfalls and what not to do: mitigating bias in ai, The British Journal of Radiology 96 (2023) 20230023.
(a) (b) Opportunity (c) Figure 4: Fairness Assessment for Cancer Prediction dataset (a) Statistical Parity (b) Propensity Score, (c) Equal