Chronic diseases pose a significant global health challenge, impacting morbidity and mortality rates. Early detection is crucial for prevention and personalised healthcare. Advanced analytics and AI offer the potential for revolutionising prediction in many field like finance [1] [2], cybersecurity [3] and in particular disease.
Supervised learning in various fields relies heavily on feature selection (FS) to reduce input dimensionality. Maintaining target class integrity amidst irrelevant characteristics is essential for accurate classification in the medical domain.
Bio-inspired optimisation emulates behaviours found in various natural creatures such as fish, insects, bird swarms, terrestrial animals, reptiles, humans, and other phenomena. These methods have been used for supervised feature selection (see [4]). The same source categorises bio-inspired optimisation algorithms into three groups based on their source of inspiration: swarm intelligence algorithms, evolutionary-based algorithms, and ecology-based algorithms. For robustness and diversity, we selected Genetic Algorithms (GA), Particle Swarm Optimisation (PSO), and Flower Pollination Optimisation (FPO), one from each category.
We refine feature subsets from medical datasets encompassing cancer, kidney, and cardiovascular diseases to enhance model accuracy and simplify data dimensionality. The aim is to improve interpretability and practicality in chronic disease prediction.
Investigating chronic diseases presents significant challenges in the healthcare domain. This study aims to improve the predictive accuracy of chronic diseases by employing machine learning (ML) and feature selection (FS) techniques, which involve data collection, preprocessing, and performance assessment.
The paper proceeds with an outline of the methodology in Section 2. Section 3 presents experimental findings, followed by a discussion in Section 4. Finally, Section 5 summarises key findings, limitations, and future directions.
Preprocessing techniques, including transformation, cleaning, imputation, balancing, and normalization, were applied to ensure data quality [5]. Subsequently, feature selection was performed by GA, PSO, and FPO algorithms. The selected features were then used for classification using Decision Trees (DT), Random Forest (RF), Logistic Regression (LR), Support Vector Machines (SVM), and K-Nearest Neighbour (KNN). Finally, we evaluated the performance of these models using various metrics.
Breast Cancer dataset: From the University of Wisconsin, this dataset involves cytological examinations to distinguish between benign and malignant tumours. It contains 569 samples and 31 features.
Kidney Disease: Medical information on chronic kidney disease, collected over two months in India, is included in this dataset, available on Kaggle or UCI. It consists of 400 samples and 25 features.
Heart failure dataset: Comprising medical records of heart failure patients during follow-up, this dataset contains 299 samples and 13 features.
Each dataset has the "diagnosis" column with binary values used as targets for supervised learning of classifiers, where 0 denotes a negative and 1 indicates a positive outcome, respectively.
Missing Values Imputation. Addressing missing data poses risks of performance degradation and biased results. We used the K-Nearest Neighbors (KNN) algorithm, known for its adaptability to diverse data types, to fill the lack in the datasets.
Data Balancing. To balance the datasets is a critical concern due to the struggle of the classifiers when faced with disparate class distributions, leading to biased models. To mitigate this issue, we used the Synthetic Minority Over-sampling Technique for Nominal and Continuous features (SMOTEEN) [6] which addresses imbalanced datasets by oversampling the minority class and cleaning the majority class by combining the SMOTE and Edited Nearest Neighbors (ENN) methods.
Min-max Normalization. We applied this scaling method to normalize the datasets to a predefined range, as follows: 𝑋𝑛𝑜𝑟𝑚 = 𝑋−𝑋 𝑚𝑖𝑛 𝑋𝑚𝑎𝑥−𝑋 𝑚𝑖𝑛 , where 𝑋𝑛𝑜𝑟𝑚 represents the normalized value of the feature, 𝑋 is the original value of the feature. 𝑋𝑚𝑖𝑛 and 𝑋𝑚𝑎𝑥 denote the minimum and maximum values respectively.
Following the data preparation stage, we applied the three aforementioned bio-inspired feature selection algorithms to each of the three datasets (see section 2.1). All algorithms employ the same fitness function, with the 𝛼 value set to 0.99 to prioritize classification accuracy.
For assessing fitness, we utilized the K-Nearest Neighbors (KNN) classifier, known for its efficiency and adopted by [7], as it does not necessitate a lengthy training phase. A neighbour count of 𝐾 = 10 was used. The feature selection algorithms were configured with 20 agents (individuals) and 100 generations.
For each dataset detailed in section 2.1, every machine learning model is trained using 70% of the data and tested using the remaining 30%, employing all features, and filtered features by PSO, FPO, and GA algorithms. This process is iterated 100 times with each iteration involving shuffling the dataset. Moreover, for each iteration, the dataset is split into training and testing sets to evaluate measures such as Accuracy, Recall, Precision, and F1-score.
Figures 1, 2, 3 show the fitness trends of the FS algorithms, and table 1 summarizes the performance of these FS algorithms in terms of feature reduction.
In table 2, we report the accuracies of the classifiers with the features. Whereas, in table 3, we report the percentage variations of the training time before and after the FS.
The final features selected by the various algorithms are:
The final features selected by the three algorithms are:
• FPO: ['anaemia', 'diabetes', 'smoking']
• GA: ['platelets', 'serum sodium', 'time'] • PSO: ['platelets', 'serum creatinine'] we note that GA has acheived a better fitness with respect to FPO, even both have achieved the same reduction percentage in dimentionality with breast cancer dataset. The two algorithms have selected different sets of features. The genetic algorithm significantly higher fitness with respect to FPO and PSO even though the the dimentionality reduction is almost the same. RF, DT, SVM and KNN have achieved a better performance on this dataset when combined with GA algorithm. In general, however, the training times have all decreased, with the maximum decrease 57% by LR model with both GA and PSO. See further metrics in figures 7, 8 , 9.
The final features selected by the various algorithms are:
• FPO: ['su', 'rbc', 'pcc', 'pe', 'ane'] • GA: ['rbc', 'bgr', 'sod', 'hemo', 'pcv', 'dm', 'cad'] • PSO: ['age', 'su', 'rbc', 'pc', 'bgr', 'sod', 'pot', 'hemo', 'pcv', 'rc', 'cad'] In this dataset, high performance was achieved with most models combined with PSO and GA, while with FPO there was a significant decrease in performance. There has been a 55% decrease in processing time without loss in the performance with LR model combined with PSO and a reduction in processing time up to 57% with LR combined with GA with a very slight reduction in the performance. The highest fitness was achieved by GA with 7 features (70% reduction in dimentionality), while the lowest fitness was achieved by FPO with the highest features reduction. See further metrics in figures 10, 11, 12.
From Table 2, it is evident that GA emerged as the most effective FS technique in terms of performance. It consistently improved accuracy across various ML models and datasets, or maintained accuracy levels compared to pre-FS values with other techniques. The accuracy enhancement with GA ranged from 0.1% to 7%. Following GA, PSO ranked second in terms of accuracy perfor- mance among the three bio-inspired algorithms. While PSO did not notably enhance accuracy, it also did not lead to significant decreases. FPO exhibited diverse outcomes across different ML models and datasets. While accuracy decreases were marginal (less than 2.5%) for most ML models on the breast cancer dataset, there were more pronounced decreases on the heart failure and kidney disease datasets.
In terms of training times, the impact was particularly notable for DT and LR, as evidenced in Table 3. Generally, training times decreased across all models when employing feature selection (FS), except for K-Nearest Neighbours (KNN) with breast cancer and kidney disease datasets, where a significant increase of up to 21% was observed. Minor fluctuations within ±2% in training times were considered insignificant, likely due to variable hardware conditions and software factors. Overall, machine learning (ML) models exhibited reduced training times with FS, especially DT and LR models with GA and PSO. The most substantial reduction in training time, up to 67%, was achieved by the LR model with FPO on the Kidney disease dataset. Although FS did not significantly improve ML model performance in most cases, and even led to a decrease in performance in some instances, the noteworthy decrease in processing times without significant loss in accuracy represents a significant achievement.
The experimental findings indicate that the GA outperformed other FS algorithms in terms of precision, recall, and F1-measure. GA demonstrated superior performance when paired with nearly all ML models compared to FPO and PSO across all datasets. However, the PSO algorithm, when combined with the LR model, exhibited slightly higher recall and F1 scores for breast cancer and kidney disease datasets, as well as marginally improved recall for heart failure dataset. Conversely, FPO generally exhibited the poorest performance when paired with various ML models. Although FPO achieved the highest recall when combined with the LR model on the heart failure dataset, its overall performance was inferior. In terms of fitness trends, GA displayed the most favourable results, with PSO closely trailing behind, while FPO yielded significantly lower fitness levels compared to GA and PSO. Further experiments are planned to investigate the behaviour of these FS algorithms with varying parameters, datasets, and ML models.
Our experiments have highlighted the importance of feature selection (FS) in improving the performance of machine learning (ML) models. The impact of FS varies depending on factors such as the chosen FS algorithm and dataset characteristics [8]. FS holds the potential to significantly enhance ML outcomes, especially for datasets with a large number of features. For example, in the breast cancer dataset, reducing features from 30 to 12 or 8 resulted in up to a 50% reduction in training time, while maintaining the same performance across various ML models. However, the effect of FS on training time