Globally, breast cancer ranks as the most widespread form of cancer among women. Machine Learning and Deep Learning approaches provide more effective means for detecting and managing this condition when compared to conventional detection methods. Advanced deep learning techniques, including LSTM (Long Short-Term Memory), Gate Recurrent Units (GRU), and Deep Belief Networks (DBN) have been used for classification of cancer. In this paper, the publicly available breast cancer dataset namely Wisconsin dataset is employed to investigate the efficacies of these deep learning techniques for classification of breast cancer. Further, the network architecture parameters are tuned for achieving better results using one of the latest swarm intelligence technique namely Sea Horse Optimization. Success rate of 95.61%, 96.49% and 98.24% respectively have been achieved by the proposed SHO-LSTM, SHO-GRU and SHO-DBN models when applied to the Wisconsin dataset.
Breast cancer happens to be the most common cancer type among women globally.
According to the Global Cancer Observatory (GLOBOCAN 2020) report [
Various deep learning techniques e.g., MLP, RNN, LSTM, GRU and DBN play crucial roles
in breast cancer detection by capturing context and features using sequential data. LSTM
and GRU are two variants of Recurrent Neural Networks (RNN) that use gating mechanism
to selectively update information overtime, with GRU being simpler and faster than LSTM
[
In Machine Learning, selection of optimal values of various parameters of a model is
termed as hyper-parameter tuning. Hyper-parameters are basically the configuration
settings that control the learning process of a model, such as learning rate, the number of
neural network layers, model architecture, batch size, activation functions, etc. specific to
the model employed. Various strategies such as grid search and random search,
metaheuristic optimization, swarm optimization, Bayesian Optimization, etc. are generally used
as optimization technique in hyper-parameter tuning. Among these approaches, swarm
intelligence plays a significant role in achieving optimal solutions and reducing detection
time for efficient control of disease. Sea-horse optimization (SHO) is a new swarm
intelligence-based meta-heuristic approach rooted in swarm intelligence, drawing
inspiration from the captivating behaviors observed in sea horses in their natural
environment [
The structure of this paper is organized as follows: Section 2 offers a concise literature review covering diverse techniques employed for breast cancer detection using ML, DL and hyper-parameter optimization. Section 3 delves into details regarding the dataset employed in this study. Methodology is elucidated in Section 4, followed by a discussion of the results in Section 5. Lastly, conclusions based on the findings are drawn in Section 6.
A brief overview of a few relevant research works carried out in the field for detection of breast cancer using various techniques has been presented in this section.
The authors of [
In [
The researchers of [
The paper [
The study outline in [
The paper [14] proposes an Enhanced Sea Horse Optimization (ESHO) combined with sine-cosine and Tent Choatic Mapping to adaptively tune the ResNet-50 parameters and optimize its performance on two agricultural image datasets: jade fungus and corn diseases. The ResNet-50 model optimized by ESHO achieves an accuracy of 96.7% for corn disease image recognition and 96.4% for jade fungus image recognition. In [15], a novel hybrid PSO-SHO algorithm combining the advantages of PSO and SHO is proposed. The proposed approach strives to minimize the real power losses and the voltage deviation of the power system by optimizing the generator voltages, transformer tap settings and reactive power compensators.
The dataset that is employed is the Wisconsin Breast Cancer Dataset (WBC) [16]. The WBC dataset consists of 569 samples of breast cancer patients with a distribution of 357 benign and 212 malignant cases. 30 characteristic features are quantified from digitized images of breast masses. They are numerical measures of attributes like cell nuclei, means, standard errors and worst values of texture, perimeter, radius, smoothness, compactness, area, concavity, symmetry, concave points and fractal dimension. The classifying labels are denoted by 0 (benign) and 1 (malignant) under the attribute ‘diagnosis’.
The data preprocessing involves cleaning, scaling features, and encoding labels for the target variables. Subsequently, it is portioned into training and testing sets, with 80% allocated for training data and 20% for testing data. Further, reshaping is used of the training and testing feature arrays to add an additional dimension representing number of channels. LSTMs and GRUs expect input data in a specific 3D shape, typically represented as (batch_size, timesteps, and features). For deep learning models, particularly those using RNNs, input data should be reshaped to meet the input requirements of the model [17].
In this paper, Gate Recurrent Unit Networks (GRU), Long Short-Term Memory (LSTM) and Deep Belief Networks (DBN) have been used to predict breast cancer. Further, we have designed these models with optimized parameters for our classification problem. In order to achieve this, we have used the Seahorse Optimization algorithm to optimize the hyper-parameters and achieve better accuracy. The focus of optimization is on various parameters involved in these neural networks, like learning rate, filter numbers, neurons, epochs etc. The fitness function optimizes based on the classification accuracy. The figure 2 demonstrates a flowchart of our proposed method.
The process begins with by selecting a deep learning mode, such as GRU, LSTM, or DBN. The input data is preprocessed. Seahorse Optimizer’s (SHO) parameters like epochs, population size are initialized along with the Objective function that optimizes based on the model’s accuracy. SHO then generates hyper-parameters for the chosen model, and the respective model is trained with these hyper-parameters. These steps are repeated till the maximum number of iterations is completed. Overtime, SHO produces the highest global accuracy, and we subsequently assess the model’s performance using various traditional performance metrics based on the parameters associated with this accuracy. The models and optimizer’s involved are discussed in detail in the coming sections.
Long Short-Term Memory (LSTM) is a deep learning model that addresses the challenges of learning long-term dependencies [18], which traditional recurrent neural networks struggle with. They are specialized neural networks designed to handle sequential data and learn long-term dependencies. LSTMs extend the basic RNN cell [19]. The basic RNN cell takes input at each step, computes a hidden state based on this input and its previous hidden state. The output from the cell helps in training and prediction. Instead of this simple cell, an LSTM cell contains three key components: (1) Input Gate: This gate controls how the amount of new information enters the cell, (2) Forget Gate: Determines which information from the previous hidden state needs to be forgotten, (3) Output Gate: Sets the output based on the current input and hidden state. Additional to the hidden state (similar to RNN’s hidden state), LSTMs have a cell state which is essentially a separate memory component that stores long-term information. Cell state is updated with the combined information of input, forget and output gates.
Gated Recurrent Units (GRUs) stem from RNNs, incorporating a gating mechanism
initially introduced in [20]. Similar to LSTMs, GRUs employ gating mechanism to
selectively incorporate or forget certain features, albeit lacking an output gate, thereby
resulting in fewer parameters compared to LSTMs. GRU also finds use in processing
sequential data such as text, speech and time-series data. GRU can control the flow of
information from previous activation state while computing the new activation state [
Deep Belief Network (DBN) is a generative graphical model, composed of multiple
layers of latent variables or more popularly known as “hidden units”. They have
connections between the layers but no between units within each layer [19]. Consider
visualizing these systems as intricate, multi-layer networks with each layer processing
information from the preceding one, progressively constructing a sophisticated
comprehension of the entire dataset. DBNs are built by layering simple, unsupervised
networks such as Restricted Boltzmann Machines (RBMs) or auto encoders. The
configuration of the output layer in a DBN is contingent upon the specific task at hand. For
instance, in a classification task involving k classes, the DBN would utilize k SoftMax units,
each dedicated to one class [
Hyper-parameters are external configuration variables set by programmers to operate model training. They are parameters that define the details of learning process. Examples of hyper-parameter optimization encompass various instances such as learning rate (which regulates the magnitude of steps taken during gradient descent optimization), batch size (which dictates the quantity of training examples utilized in each iteration of gradient descent), the number of hidden layers, activation functions (like ReLU, sigmoid, tanh), dropout rate (indicating the fraction of neurons within dense layers, among others. Hyper-parameter optimization (HPO) is the process of selecting optimal values for a machine/deep learning model’s hyper-parameters. HPO can be seen as the last step of model design and the initial step of neural network training [22]. It finds a tuple of hyperparameters that gives an optimal model with enhanced accuracy or prediction. Over the years many techniques have been employed for hyper-parameter optimization such as Grid Search, Random Search, Bayesian Optimization, Genetic Algorithms and even swarm based techniques like Particle Swarm Optimization and Ant Colony Optimization [23]. For the optimization of DL models proposed in this study we have taken the hyper-parameters as represented in table 1, along with the new Seahorse Optimization Algorithm.
Introduced in 2022, the Seahorse Optimization (SHO) algorithm represents a novel
swarm-based meta-heuristic optimization approach [
The following equations (1) and (2) denote the spiral and Brownian movement behaviors of seahorses respectively. SHO utilizes Lévy flight to emulate the spiral movement observed in seahorses, which aids in preventing SHO from becoming trapped in local optima. In the spiral movement, the three dimensional vector of coordinates (x, y, z) is denoted by x, y and z. Regarding the Brownian equationl denotes the constant coefficient and βt represents the coefficient for the motion random walk.
Equation (3) is the mathematical representation of predation, withr2representing a random number generated by SHO to distinguish between success and failure scenarios. If r2exceeds 0.1, the predation by the seahorse is deemed successful; otherwise, it results in failure. α denotes the step size of the seahorse’s movement in pursuit of prey. 2 ( + 1) = ( ) = { × ( −
× 1 ( )) + (1 − ) × ,
(1 − ) × ( 1 ( ) −
×
) + × 1
( ),
seahorses. Xs2ort is fitness values of the population in ascending order. (6) denotes the
mathematical equation representing an offspring. r3 is a random number between [
(1: ⁄2)
ℎ = 2 ( ⁄2 + 1: ) = 3
ℎ + (1 − 3) ℎ ( + 1) = ( ) +
(λ)(( ( ) − ( ) × × × + ( ))
X1new(t + 1) = Xi(t) + rand× l × βt × (Xi(t) − βt × Xelite)
The algorithm starts with a randomly generated population of seahorses, with each seahorse representing a potential solution. It uses a normal distribution to decide among the two movement behaviors of seahorses. It mimics the high success rate of seahorses in hunting to enhance exploitation capabilities. It draws inspiration from the breeding behavior of seahorses to generate new solutions, hoping to improve upon the current best solution. If the hunt is successful, the seahorse (problem) moves towards the prey (best solution) otherwise search space exploitation continues. The algorithm followed by SHO is explained in the following flowchart:
The SHO algorithm maintains a balance between exploration (diversification) and exploitation (intensification) to mitigate the risk of getting stuck in local optima and to efficiently locate the global optima. This algorithm has been applied to various engineering design problems, and shows promising results.
Various Performance metrics namely accuracy, F1-score, recall, precision and specificity have been used for comparison of results between standard LSTM, GRU and DBN and their SHO optimized counterparts and these parameters are given in the following equations (7)-(11).
1 = + + + + 100 = +
= 2 × =
Accuracy is the ratio of correctly classified instances to the aggregate instances. TP (True Positive), TN (True Negative), FP (False Positive) and FN (False Negative) are the number of correctly and incorrectly classified positive and negative cases. Sensitivity or recall is the fraction of positive cases that the classifier correctly identifies, whereas specificity is the fraction of negative cases that a classifier correctly identifies. Precision is the proportion of true positives out of all predicted positives. F1-score is a measure of performance that combines precision and recall as it is the harmonic mean of the two metrics.
The results obtained from DL models with and without SHO optimization are presented in the following Table 2 and Table 3 respectively.
Before optimization, the LSTM and GRU models achieved accuracies of 92.10% and 93.85%, respectively, while the DBN model achieved an accuracy of 92.11%. However after the hyper-parameter optimization with SHO, the performances of all models significantly improved. The SHO-LSTM achieved an accuracy of 95.61%, SHO-GRU demonstrated an accuracy of 96.49% and SHO-DBN acquired an accuracy of 98.25%.
Furthermore, the SHO-DBN model consistently outperformed the other models across all metrics, demonstrating its effectiveness in breast cancer detection. Notably, the SHOGRU model achieved perfect recall (100%), indicating its ability to correctly identify all positive cases of breast cancer. GRU demonstrated better performance than LSTM across all metrics with or without optimization, indicating its greater efficiency due to lesser number of model parameters. The graphical comparisons of performance metrics for the un-optimized and optimized DL models are presented in Figure 4(a), 4(b), and 4(c).
Machine learning and deep learning techniques offer more efficient ways to detect and manage breast cancer compared to traditional methods. Advanced DL models, namely LSTM, GRU and DBN have been successfully applied to Wisconsin dataset for breast cancer classification. Additionally, network architecture parameters are fine-tuned using the state-of-the-art swarm intelligence technique known as Sea Horse Optimization. In this study, we explored the effectiveness of these deep learning techniques for achieving improved performances when applied to two different breast cancer datasets. Specifically, the SHO-DBN model emerges as the most effective model for this task, with the highest accuracy, precision, specificity and F1-Score. These findings underscore the potential of optimized deep learning models as valuable tools in the early detection and management of breast cancer. Histopathological Breast Cancer Diagnosis”, Cancers, vol. 15, p. 885, 2023, doi: 10.3390/cancers15030885. [14] Z. Li, S. Qu, Y. Xu, X. Hao, N. Lin, “Enhanced Sea Horse Optimization Algorithm for Hyperparameter Optimization of Agricultural Image Revolution”, Mathematics, vol. 12, no. 3, p. 368, 2024, doi:10.3390/math12030368. [15] H. M. Hasanien, I. Alsaleh, M. Tostado-Véliz, M. Zhang, A. Alateeq, F. Jurado, A. Alassaf, “Hybrid particle swarm and sea horse optimization algorithm-based optimal reactive power dispatch of power systems comprising electric vehicles”, Energy, vol. 286, pp. 129583, 2024, doi:10.1016/j.energy.2023.129583. [16] “Diagnostic Wisconsin Breast Cancer Database”, UCI Machine Learning Repository.
Available: https://archive.ics.uci.edu/dataset/17/breast+cancer+wisconsin+diagnostic. [17] M. Chhetri, S. Kumar, P. P. Roy, B.-G. Kim, Deep BLSTM-GRU “Model for Monthly Rainfall Prediction: A Case Study of Simtokha”, Bhutan, Remote Sensing, vol. 12, no. 19, p. 3174, 2020, doi:10.3390/rs12193174. [18] A. Sherstinsky, “Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network”, Physica D: Nonlinear Phenomena, vol. 404, p. 132306, 2020, doi: 10.1016/j.physd.2019.132306. [19] G. E. Hinton, “Deep belief networks”. In E. N. Zalta (Ed.), Scholarpedia, vol. 4.5, p. 5947, 2009, doi:10.4249/scholarpedia.5947. Available: http://www.scholarpedia.org/article/Deep_ belief_networks. [20] J. Chung, C. Gulcehre, K. Cho, Y. Bengio, “Empirical evaluation of gated recurrent neural networks on sequence modeling”, arXiv preprint, arXiv: 1412.3555v1, 2014, doi: 10.48550/arXiv.1412.3555. [21] Y. Bengio, P. Lamblin, D. Popovici, H. Larochelle, “Greedy layer-wise training of deep networks”. In B. Schölkopf, J. Platt, T. Hoffman (Eds.), Advances in Neural Information Processing Systems, MIT Press, vol. 19, pp. 153-160, 2006. Available: https://deeplearning.cs.cmu.edu/ pdfs/Bengio_et_al_NIPS_2006.pdf. [22] T. Yu, H. Zhu, “Hyper-Parameter Optimization: A Review of Algorithms and Applications”, 2020. Available: https://arxiv.org/abs/2003.05689. [23] W.-C. Yeh, Y.-P. Lin, Y.-C. Liang, C.-M. Lai, X.-Z. Gao, “Simplified Swarm Optimisation for the Hyperparameters of a Convolutional Neural Network”, 2023. Available: https://arxiv.org/ftp/arxiv/papers/2103/2103.03995.pdf. [24] H. Alahmer, A. Alahmer, M. I. Alamayreh, M. Alrbai, R. Al-Rbaihat, A. Al-Manea, R.
Alkhazaleh, “Optimal water addition in emulsion diesel fuel using machine learning and sea-horse optimizer to minimize exhaust pollutants from diesel engine”, Atmosphere, vol. 14, p. 449, 2023, doi: 10.3390/atmos14030449.