Bias in Artificial Intelligence (AI) models refers to AI systems that produce biased results that reflect and amplify human prejudices within a community, encompassing past and contemporary social injustices [1]. These biases when replicated in medical datasets can have life-threatening consequences due to incorrect diagnosis or treatment recommendations [2,3]. For example, German researchers built a skin cancer detection system using neural networks in 2016. The system was able to detect 95% of melanoma cases accurately. It was trained on 10,000 skin images and outperformed 58 dermatologists. Later, it was found that the data was highly dominated by white skin images and did not generalise well to a diverse population [4]. These biases can be handled at pre-processing, algorithmic level or in the post-processing stages of an AI model development [5]. Handling bias would help achieve fair models that do not discriminate against different groups and treat them unfairly [6]. Pre-processing techniques involve handling bias at the data level. One of the widely used techniques to mitigate bias is over-sampling. Over-sampling is the generation of synthetic data that mirrors the characteristics of real-world data. It helps to reduce bias by balancing the representation of different demographic groups so that machine learning models produce reasonable outcomes and generalise well over a diverse population [7]. There are several techniques to generate synthetic data to ensure fairness in medical datasets [8]. These techniques include SMOTE [9], FairSMOTE [10], BorderlineSMOTE [11], and Cluster-based over-sampling [12]. Moreover, deep learning is also widely used to generate artificial data because of its high efficiency and accuracy in generating data. The most commonly used algorithm is the Generative Adversarial Network (GANs) that have gained immense popularity in the research community [13].

GAN is a deep learning model that mainly consists of two neural networks: a Generator used to generate artificial data and a discriminator that tries to distinguish between real and synthetic data to improve quality. These models were first introduced to process only image data, but later different variants of GAN were proposed to process tabular data as well. These variants include Tabular GAN (TGAN) [14], Conditional Tabular GAN (CTGAN) [15], Medical GAN (MEDGAN) [16], Multi-Categorical GAN (MC-MedGAN) [17] and many more.

In this study, we evaluated various GAN variants including GAN, TGAN, CTGAN, MedGAN, and MC-MedGAN to generate synthetic samples to balance different group representations within medical datasets. The newly balanced dataset was fed into different ML models including Logistic Regression (LR), Random Forest (RF), Decision Tree (DT), and K-Nearest Neighbour (KNN) to draw a comparison. The GAN models are evaluated on three different medical datasets that consist of gender as a sensitive attribute to balance: the Asthma Disease Dataset [18], the Heart Disease Prediction Dataset [19], and the Cancer Prediction dataset [20]. The performance is evaluated using various metrics i.e., accuracy, precision, F1-score, recall, and Area Under Curve (AUC) scores. Fairness is evaluated using Equal Opportunity (EO) [21], Propensity Score (PS) [22], and Statistical Parity (SP) [23].

In today's world, AI is an integral part of the healthcare system. The AI model must incorporate transparency and accountability. The goal of this research is to reduce bias in medical datasets that contain inherent biases due to unequal representation of different demographic groups. AI models can become unfair and imbalanced, particularly in the healthcare sector, where underrepresented groups may receive scant care. Bias in medical datasets poses a significant challenge to the reliability of predictive models [24]. This could be critical for healthcare systems since an automated model prediction has a direct effect on patients that affects their mental health, and quality of life or may risk the life of an individual [25] as well it also leads to financial loss [26]. Due to an unbiased dataset, certain populations may receive incorrect diagnoses or treatments as a result of unreliable predictions brought on by bias in datasets. Nonetheless, GANs provide a potentially helpful way to generate AI data that can assist in balancing underrepresented groups in health databases. The aim to explore how GAN-based techniques can eliminate bias through data augmentation and enable more reliable and equitable Machine Learning (ML) models motivates this effort [13]. The comparative study's main goal is to identify the optimal variant to lessen bias in medical datasets. We want to improve the quality of treatment by lowering bias and ensuring that AI systems generate reliable, accurate, and equitable forecasts for a range of demographics. Therefore, the motivation of this study is to investigate different variants of GAN including TGAN, CTGAN, MedGAN, and MC-MedGAN for their efficacy in mitigating bias and improving predictive performance on multiple medical datasets. This work will serve as a foundation for further experimentation on data generation via GAN to mitigate biases.

Hypothesis: GAN-based data generation methods can help to reduce biases and ensure fairness in medical datasets.

The formulated research questions to explore the above hypothesis are as follows:

• Does GAN-based synthetic data generation help reduce biases in medical datasets? If yes, which GAN variant performs better among basic GAN, TGAN, CTGAN, MedGAN, and MC-MedGAN?

The rest of the article is structured as follows: Section 3 highlights some of the recent related work. Section 4 explains the methodology in detail. Section 5 explains the results. Section 6 discusses the hypothesis and research questions and Section 7 concludes the discussion with future work.

GANs have gained significant attention in recent years due to their capability of generating highquality data. Therefore, this section reviews the recent methodologies that leverage GAN models to generate synthetic data. A study [27] presents the potential of GANs in generating synthetic data from observational health data and discusses some of the unique challenges associated with healthcare datasets, such as concerns about class imbalance. Observational Health Data (OHD) is highly valuable for medical research and health informatics. The use of such data is severely limited because of strict regulations. It highlights that GAN-generated synthetic data can help overcome some of the common challenges, such as bias, privacy and class imbalance. The authors argue that GANs are useful in generating healthcare data to combat the scarcity of high-quality medical datasets. Moreover, to address the challenges of drift and class imbalance of gas detection systems, [28] employed CTGAN for data augmentation. The result shows a significant improvement in the classification accuracy of each class for both Support Vector Machine (SVM) and Multi-Layer Perceptron (MLP) thus reducing bias toward the majority class. They conclude that CTGAN provides a feasible solution to generate a balanced dataset.

In another study [29] various variants of GAN including CTGAN, TGAN, and Wasserstein GAN (WGAN) are utilised for the anonymisation of real data through data synthesis. These models were compared for precision, recall, and coverage scores to evaluate the generation of realistic tabular data, handling missing and class-imbalanced data, and ensuring privacy. The results show that, although no GAN method performs best in each evaluation metric, CTGAN and TGAN produce better scores in most of the evaluation metrics. Additionally, in [30] a new variant of GAN called Multi-label Timeseries GAN (MTGAN) is proposed to generate sequential Electronic Health Record (EHR) data using a gated recurrent unit with a smooth conditional matrix, while the critic evaluates temporal features using Wasserstein distance for improving the quality of synthetic data. The results show that MTGAN generates realistic EHR data effectively and improves accuracy for uncommon diseases.

The above studies show that GANs have the potential to generate high-quality diverse datasets that can be used to handle bias in real-world datasets. Therefore, to analyse the capabilities of different GAN variants, this study aims to conduct multiple experiments and then assess the fairness within the newly generated synthetic medical datasets.

Figure 1 shows the systematic methodology diagram used to evaluate the different variants of GANs. First, the data is preprocessed and split into standard train and test sets. Then, the data is fed into the GAN variant to generate synthetic data. The newly generated data is augmented with the real data to balance the number of samples for the sensitive attributes and the output label. Afterwards, ML models are trained on the newly generated data to evaluate the overall performance as well as the fairness of the models.

The data preprocessing includes one hot encoding to replace categorical variables with numerical numbers. Afterwards, we applied z-score normalisation on each distinct numerical feature because they did not contain extreme outliers [31]. Normalisation helps to specify each variable within a specified range to simplify the model-learning process [32]. Then, the resulting dataset is split into a 70:30 ratio for train and test sets.

In order to balance the dataset for the sensitive attribute i.e., Gender and class labels. We employed five GAN architectures: basic GAN, TGAN, CTGAN, MedGAN and MC-MedGAN. These variants are specifically designed to handle tabular and medical datasets which is the primary focus of our study. GAN is a type of neural network architecture where two networks, a generator, and a discriminator, are trained simultaneously [33,34,35]. Tabular GAN is an application-driven variant of the GAN that is designed to generate synthetic tabular data, containing rows and columns like in a spreadsheet or database [33,14,36]. The CTGAN is an extension to Tabular GAN that generates synthetic tabular data while taking into consideration the distribution of dependent target variables. This will help associate relations between columns and observe dependence relationships [15]. MedGAN is a specialised version of GAN that generates synthetic data in the medical field, mainly in tabular form containing sensitive information [16]. MC-MedGAN is a variant of MedGAN designed for handling multi-categorical variables, commonly present in medical datasets [17].

After generating synthetic samples to balance the datasets for sensitive attribute (gender) and class labels, different commonly used ML classifiers including Logistic regression (LR), Random Forest (RF), Decision Tree (DT), K-Nearest Neighbour (KNN) with default parameters are trained on the newly generated datasets to evaluate the performance of GAN variants.

To evaluate the performance of GAN variants, we used three different medical datasets that contain sensitive attributes. The details of each of these datasets are as follows:

Asthma Disease Dataset: The Asthma Disease Dataset [18] contains a record of 2,392 samples with 28 features. The output label is the diagnosis indicator, which is taken as 0 for the absence and 1 for a positive case. It contains 2,268 samples for class 0 as compared to 124 samples with class label 1. Also, the number of samples for males is 1212 whereas for females the count is 1180.

Heart Disease Prediction Dataset: The Heart Disease Prediction Dataset [19] consists of 13 features and 303 samples. The dataset contains 207 male and 96 female samples.

The Cancer Prediction Dataset [20] contains 1,500 samples with 8 features. The target variable 'diagnosis' indicates whether a patient has cancer or not (0 for no cancer and 1 for cancer). The diagnosis distribution shows 943 patients without cancer and 557 with cancer. There are 736 female samples and 764 males in total.

The performance is evaluated by training different ML classifiers as mentioned in Section 4. The classifiers are assessed using accuracy, f1-score, precision, recall and AUC. Whereas the fairness of the dataset is evaluated via EO, PS, and SP. EO guarantees that all individuals receive the same treatment and meet the same requirements [21]. PS can be defined as the conditional probability of being exposed to a treatment given the observed covariates [22]. SP is a fairness criterion that requires the probability of a favourable outcome to be the same for each demographic group [23]. Tables 1, Table 2, and Table 3 show each classifier's performance on the original as well as on each generated dataset. It can be seen that MedGAN performs well for the Asthma Disease Dataset and Cancer Prediction Dataset while MC-MedGAN has a better score for the Heart Disease Dataset. Figure 2, shows the fairness metric performance on the Asthma Disease dataset. The SP, PS, and EO scores improve when the dataset is balanced for class label and gender. MEDGAN has a better performance for all three datasets followed by MC-MedGAN and TGAN. The same performance is observed for the other two datasets. The other graphs are given in Appendix A.

Overall, the results show that balancing the dataset for class labels and sensitive attributes improves the performance as well as the fairness of the model. Among different GAN variants, the MEDGAN produces good results and lower statistical, propensity and equal opportunity scores showing its great capability for reducing bias followed by MC-MedGAN. Moreover, the predictive ability of RF classifiers is better than other classifiers in terms of accuracy, precision, recall, f1-score, and AUC.

This section discusses the overall findings of the study in view of the literature review and extensive experimentation conducted to analyse our hypothesis. Based on our research question, the experiments show that classifier performance as well as the fairness metrics score improves when the datasets are balanced for sensitive attributes and class labels. Figure 2 shows the improvement in the fairness scores across each metric when the dataset is balanced via synthetic data generation using GAN variants as compared to the original dataset. Moreover, the analysis of each GAN variant based on performance evaluation using accuracy, precision, F1-score, recall, AUC and fairness metrics via EO, PS, and SP indicates that the MedGAN produces efficient performance followed by MC-MEDGAN across all three datasets. To validate any statistically significant difference between these two methods, we applied a paired t-test on the EO, PS, and SP scores for each of these methods. The p-values for EO, PS, and SP came out to be 0.34, 0.61, and 0.30 respectively. Therefore, we fail to reject our hypothesis and conclude that these two methods are not significantly different. These GAN variants are specifically designed for medical datasets to capture the interdependencies between the different variables to generate synthetic data similar to original data properties [16,17]. However, further experimentation with other datasets including post-hoc tests will be conducted in future to provide deeper insights into the capability of GAN variants for data generation.

In this paper, we tested different types of GANs for their capacity to produce synthetic tabular data to decrease bias in medical datasets. Our key findings are GAN-based models are effective for bias migration and GAN can provide a balanced dataset to produce generalised AI models and provide a solution AI for all and AI for good. On the other hand, traditional GANs were successful but medical domain-based GANs displayed greater performance in generating high-quality and unbiased data. It drives us to have more specific models in the future. Despite certain advantages of the GAN, we face some obstacles such as evaluation metrics. There is a need to have more standardised and compressive evaluation metrics of this model focused on decreasing bias. The studies in this article suggest that synthetic data can assist in eliminating bias and improve the effectiveness of the classifier. Moreover, MedGAN performs better in terms of SP, PS, and EO. In future, we will extend our work for various variations of GAN focused on refining GAN architecture to adapt the multimodality medical data, bias-sensitive evaluation mechanism and testing the GAN-based techniques in real-world clinical data.