Heart failure is a very significant public medical concern globally, posing substantial challenges to healthcare systems due to its high prevalence, mortality rates, and associated healthcare costs. Early and accurate prediction of heart failure is crucial for effective patient management, timely interventions, and improved clinical outcomes. In recent years, the integration of advanced machine learning techniques with healthcare data has shown promising results in enhancing predictive models for heart failure prognosis. [1] This research paper explores the application of Sparrow Search Algorithm (SSA), a novel metaheuristic optimization algorithm which is inspired by the sparrows' behavioral foraging nature [2], combined with three deep learning (DL) techniques, namely Long Short-Term Memory (LSTM), Bidirectional-LSTM (Bi-LSTM) and Gated Recurrent Unit (GRU) for heart failure prediction. The chief objective of this comparative research is to investigate the efficiency of SSA in optimizing DL models to accurately predict heart failure risk.
Heart failure is a very significant public medical concern globally, posing substantial challenges to healthcare systems due to its high prevalence, mortality rates, and associated healthcare costs. Early and accurate prediction of heart failure is crucial for effective patient management, timely interventions, and improved clinical outcomes. In recent years, the integration of advanced machine learning techniques with healthcare data has shown promising results in enhancing predictive models for heart failure prognosis. [1] This research paper explores the application of Sparrow Search Algorithm (SSA), a novel metaheuristic optimization algorithm which is inspired by the sparrows' behavioral foraging nature [2], combined with three deep learning (DL) techniques, namely Long Short-Term Memory (LSTM), Bidirectional-LSTM (Bi-LSTM) and Gated Recurrent Unit (GRU) for heart failure prediction. The chief objective of this comparative research is to investigate the efficiency of SSA in optimizing DL models to accurately predict heart failure risk.
The motive of this research is to develop robust and reliable predictive models that leverage both the optimization capabilities of SSA and the representational power of DL techniques to enhance heart failure prediction accuracy. By leveraging large-scale healthcare datasets, this study is targeted to enhance the advancement of predictive analytics in healthcare, facilitating early identification of individuals at risk of heart failure and enabling proactive intervention strategies to mitigate adverse health outcomes.
The UCI Cleveland Heart Disease dataset is a widely used dataset for heart disease prediction and classification. It contains 1025 instances and 14 attributes, including both categorical and continuous variables. Some of the dataset attributes are Sex, Age, Fasting Blood Sugar, Serum Cholesterol(chol), Chest Pain Type (CP) and many others. [3] This dataset is of great utility in creating and validating models for forecasting the probability of heart disease, relying on a patient's characteristics. It has been widely employed in healthcare-focused projects involving machine learning and data analysis.
The proposed Swarm-ANN strategy, developed by Sudarshan Nandy, introduces an innovative approach to cardiovascular disease prediction by leveraging swarm intelligence for neural network optimization. The strategy involves the random generation of Neural Networks, multiple stages of weight changes, and a heuristic formulation for weight adjustment. While achieving an impressive accuracy of 95.78% on a benchmark dataset, the research may be limited in its generalizability due to the focus on specific dataset characteristics and predefined parameter ranges for learning rates and population sizes. This could pose a risk of overfitting to the peculiarities of the chosen dataset, raising questions about the strategy's adaptability to diverse datasets and real-world scenarios. [4] The presented MLP-PSO Hybrid Algorithm, introduced by Ali Al Bataineh and Sarah Manacek, contributes significantly to healthcare by leveraging machine learning for enhanced heart disease prediction. The study acknowledges the challenge of developing heart disease due to multiple underlying factors. Utilizing the Cleveland Heart Disease dataset, the suggested MLP-PSO hybrid algorithm demonstrates superiority over 10 different ML algorithms, achieving a notable accuracy of 84.61%. However, the research does not explicitly address potential limitations, leaving room for further exploration into the algorithm's scalability, robustness, and performance across varied datasets and clinical settings. [5] The methodology proposed by Shahrokh Asadi, merging multiple objective particle swarm optimization (MOPSO) as well as Random Forest for the prediction of heart disease, addresses the challenges associated with traditional diagnostic methods. The fusion of evolutionary multi-objective optimization and Random Forest aims to produce diverse and accurate decision trees simultaneously, demonstrating promising results with an accuracy of 88.26% on the Statlog dataset. Comparative analyses across six heart datasets showcase the superiority of the proposed method, emphasizing its potential to outperform conventional Random Forest algorithms with different classifiers. Nevertheless, potential loopholes may include a lack of in-depth analysis of algorithmic complexities and scalability concerns across different datasets. Addressing these aspects could further validate the proposed methodology's effectiveness and reliability in diverse clinical applications. [6] The newly proposed heart disease prediction model, QPSO-SVM, by E. I. Elsedimy, Sara M. M. AboHashish, and Fahad Algarni, showcases innovation through the integration of quantum-behaved particle swarm optimization (QPSO) and support vector machine (SVM). While achieving a remarkable accuracy of 96.31% on the Cleveland heart dataset, potential loopholes may include the need for comprehensive exploration of QPSO-SVM's computational efficiency and scalability, especially when applied to larger datasets or in resource-constrained environments. Further investigation into these aspects can enhance the model's applicability and robustness. [7] The Deep Edge Intelligence-based solution by Hossain and Tabassum employs the OQFFN algorithm on a Raspberry Pi, ensuring real-time heart failure predictions in IoT-based healthcare. The approach enhances reliability without constant network stability, making it unique in comparison to cloud-based services. Evaluation shows OQFFN's superior accuracy and efficiency at the edge, with potential applications in Ambient Assisted Living. Despite success, limitations in edge processing for complex algorithms are acknowledged, highlighting the need for future developments in distributed prediction models. The research significantly contributes to enhancing IoT-based healthcare systems. [8]
Our principal methodology of implementation is outlined as follows: Explore Python Packages for Genetic and Evolutionary Algorithms: The research began by exploring various Python packages for genetic algorithms (GAs) and evolutionary algorithms (EAs), such as DEAP, PyGAD, and Genetic Algorithm Python. Further evaluation involved assessing each package based on factors like functionality, ease of use, documentation, and community support.
Implement Mealpy Package in Python: We implemented the Sparrow Search Algorithm (SSA) from the Mealpy package, to aid in feature selection. It was done by implementing genetic, swarm-based and evolutionary patterns followed by sparrows moving in a swarm using Python capability for optimization tasks.
Feature Selection Using Sparrow Search Algorithm: We studied the SSA algorithm and its associated mathematical formulae to change the bias, weight and threshold values applied in the code so that it provides optimal feature selection for our dataset.
Apply DL models like LSTM, Bi-LSTM, and GRU: We chose the deep learning (DL) models, LSTM, BiLSTM, and GRU, based on the nature of our data and task requirements. Implemented these models using TensorFlow and PyTorch in Python. We also used libraries like Keras for easier implementation. Further data preprocessing was done, including normalization and sequence padding. We then trained each DL using hyperparameters that were fine-tuned using SSA for optimized performance.
Compare the Results: We compared the results by assessing the performance metrics. We visualized the results using plots to present comparisons between different models and algorithms.
In the pursuit of optimal performance and robustness in deep learning models, meticulous attention to data preprocessing methodologies is imperative. Data preprocessing serves as a critical precursor to model training, facilitating enhanced model generalization and efficacy by mitigating the deleterious effects of noise, imbalance, and irregularities inherent in raw data. Preprocessing steps commonly entail data normalization to standardize feature scales, imputation techniques for handling missing values, and encoding categorical variables to numerical representations. Additionally, feature selection or extraction techniques may be applied to reduce dimensionality and enhance model interpretability. In the context of our investigation, comprehensive data preprocessing procedures were meticulously executed, including outlier detection and removal to attenuate the influence of aberrant data points, and stratified sampling to alleviate class imbalance concerns. Subsequently, the preprocessed data were subjected to rigorous cross-validation to ascertain model performance and generalization capabilities. These meticulous preprocessing endeavors culminated in superior model performance metrics and bolstered the veracity of our findings.
In the context of feature selection, the Sparrow Search Algorithm (SSA) incorporates mathematical formulations that enable the optimization of feature subsets based on objective functions designed to evaluate their relevance and discriminative power. SSA employs mathematical expressions to model the movement of individual sparrows within the feature space, with each sparrow representing a potential feature subset. The algorithm utilizes mathematical operators such as random walks, levy flights, and local search mechanisms to explore and exploit the solution space efficiently. Furthermore, SSA employs fitness functions that quantitatively assess the quality of feature subsets based on criteria such as classification accuracy, information gain, or other relevant metrics. These fitness functions guide the optimization process by assigning higher scores to feature subsets that contribute positively to the performance of the machine learning model. [10][11]
In the realm of hyperparameter tuning, the Sparrow Search Algorithm (SSA) offers a versatile and efficient approach to optimize the configuration settings of machine learning models. SSA operates by iteratively exploring the hyperparameter space, represented by individual sparrows, and updating their positions based on fitness evaluations. In the context of hyperparameter tuning, SSA dynamically adjusts hyperparameter values to maximize model performance on a validation dataset. This involves formulating an objective function that quantifies the model's performance based on chosen evaluation metrics such as accuracy, loss, or cross-validation scores. SSA optimizes hyperparameters by iteratively evaluating different configurations, seeking to minimize the objective function. Through a combination of exploration and exploitation, SSA efficiently searches for optimal hyperparameter settings, adapting its search strategy based on the observed performance of candidate solutions. By leveraging SSA for hyperparameter tuning, researchers can automate the process of optimizing model configurations, thereby improving model performance, generalization capabilities, and computational efficiency. [12]
The Sparrow Search Algorithm (SSA) is a recently introduced metaheuristic optimization algorithm inspired by the cumulative foraging behavior of sparrows in searching for food. It is categorized with a family of swarm intelligence algorithms, which mirror the social behavior of organisms to solve complex optimization problems. SSA operates based on the concept of exploration and exploitation, where individual sparrows in the population search for optimal solutions through a combination of random exploration and local exploitation of promising regions in the search space. [13] From a research perspective, the Sparrow Search Algorithm offers several notable characteristics that make it appealing for solving optimization problems. Firstly, SSA exhibits strong global search capabilities, allowing it to efficiently explore the solution space and locate potential optima across a wide range of problem domains. This attribute is particularly advantageous for addressing highdimensional and non-convex optimization problems commonly encountered in various scientific and engineering fields. [14] Secondly, SSA incorporates adaptive mechanisms to dynamically adjust its search behavior based on the efficacy of solutions generated during the optimization process. This adaptability enables the algorithm to effectively balance exploration and exploitation, thereby enhancing its convergence speed and solution quality over successive iterations.
Moreover, the use of adaptive parameters reduces the reliance on manual parameter tuning, making SSA more user-friendly and accessible to researchers and practitioners. [15]
It is a simplified recurrent neural network (RNN) architecture adept at capturing temporal dependencies in serialized data. GRU operates by employing gating mechanisms to modify the network's hidden state during each time step, controlling the flow of information within the network. [16] It comprises of two gating mechanisms where the reset gate decides the extent to which the prior hidden state should be disregarded, whereas the update gate governs the degree to which the new input influences the hidden state's update. Subsequently, the GRU's output is determined based on the modified hidden state. With its update and reset gates, GRU efficiently retains and updates hidden states, making it suitable for tasks like heart failure prediction. Its streamlined design and adaptability enable effective modeling of both short-term fluctuations and long-term patterns in patient data. Additionally, the streamlined design of GRU reduces the risk of overfitting and enables faster convergence during training. [17]
LSTM represents a specific architecture within recurrent neural networks (RNNs) tailored to tackle the complexities associated with capturing prolonged dependencies in sequential data. In contrast to conventional RNNs, LSTM incorporates distinct memory cells, allowing the network to preserve information across extended temporal intervals. This design renders LSTM particularly adept at handling tasks involving sequential data, including but not limited to time series prediction, language processing, and healthcare analytics. [18] The fundamental elements of an LSTM unit comprise the input gate, forget gate, output gate, and cell state, each playing a crucial role in managing information flow within the network. The input gate regulates what information should be stored in the cell state, while the forget gate determines which information should be discarded from the cell state. Through the cell state, the LSTM unit retains information over time, facilitating the capture of long-term dependencies. Lastly, the output gate governs which information should be passed from the cell state to the subsequent time step. Renowned for its efficacy in modeling temporal dependencies and mitigating challenges such as vanishing gradients, the LSTM has garnered widespread adoption across diverse domains. [19]
It is an expansion of the classic Long Short-Term Memory (LSTM) design, aiming to grasp context from both past and future in sequential data. It is composed of two LSTM layers: one handling the input sequence forwards and the other backwards. This bidirectional approach allows the model to grasp dependencies from both preceding and succeeding time steps, thereby improving its comprehension of temporal sequences and predictive accuracy. [20] In Bi-LSTM, each hidden state incorporates input not just from the past but also from the future, enabling the model to adeptly handle long-term dependencies. This bidirectional method proves particularly advantageous in tasks necessitating context from both past and future, such as natural language processing, speech recognition, and time series analysis. The Bi-LSTM architecture comprises forward and backward LSTM layers linked to a dense layer, amalgamating insights from both directions prior to making predictions. By harnessing information from both preceding and succeeding contexts, Bi-LSTM models excel in capturing intricate patterns in sequential data, outperforming unidirectional LSTM models. [21]
In this paper, we conducted experiments that are aimed to investigate the effectiveness of Sparrow Search Algorithm (SSA) combined with three Deep Learning (DL) models, namely Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), and Bidirectional Long Short-Term Memory (Bi-LSTM), for heart failure prediction. Among these models, Bi-LSTM emerged as the most accurate predictor of heart failure risk with the accuracy of 97.86%, while GRU and LSTM got 82.92% and 97.50% respectively. As for the models that were not optimized by swarm techniques, showed comparatively less accuracy i.e. 77.17%, 89.75% and 90.73% were the results of the DL models GRU, LSTM and BI-LSTM respectively.
The results revealed that all the swarm integrated models outperformed various normal deep learning models in terms of predictive accuracy, precision and recall. Among all the swarm optimized DL models with SSA, Bi-LSTM showed the best result i.e. 97.86% accuracy. It's ability to grasp both past and future sequential data proved to be improving the ability of the model to understand the complex temporal patterns underlying heart failure progression. This bidirectional processing enabled Bi-LSTM to leverage information from both preceding and succeeding time steps, leading to more accurate predictions compared to unidirectional models.
The integration of Sparrow Search Algorithm (SSA) with DL models significantly enhanced the predictive performance of Bi-LSTM. SSA effectively fine-tuned the various parameters of the Bi-LSTM model, enabling it to achieve superior accuracy in heart failure prediction tasks. The optimization process facilitated the exploration of the solution space and the identification of optimal model configurations, leading to improved generalization and robustness.
While Bi-LSTM demonstrated remarkable accuracy in heart failure prediction, further research is warranted to explore its applicability in real-world clinical settings. Future studies could focus on evaluating the interpretability of Bi-LSTM models, exploring the influence of different input features on predictive performance, and conducting prospective validation studies to assess the model's clinical utility. Additionally, exploring ensemble techniques and hybrid models combining DL with other machine learning approaches could further enhance predictive accuracy and robustness. prediction. Through extensive experimentation and evaluation on real-world healthcare datasets, we have demonstrated the effectiveness of the proposed approach in accurately predicting heart failure risk.
The results obtained from our experiments highlight the significant impact of optimization techniques such as SSA in fine-tuning DL models for improved predictive performance. We observed that the combination of SSA with DL techniques not only enhanced the predictive accuracy of heart failure prediction models but also contributed to better generalization and robustness.
Furthermore, our findings underscore the importance of selecting appropriate DL architectures for healthcare analytics tasks. While LSTM, GRU, and Bi-LSTM all exhibited promising results, Bi-LSTM, with its ability to capture both past and future context in sequential data, emerged as the most effective model for heart failure prediction in our experiments.
Overall, the outcomes of this study have implications for clinical practice, offering healthcare practitioners a valuable tool for early detection and risk stratification of heart failure patients. By leveraging the synergy between optimization algorithms like SSA and advanced DL models, we can pave the way for more accurate, efficient, and personalized healthcare interventions, finally progressing towards an improved patient health results and boosted quality of care in the management of heart failure.