Improving time series forecasting methods is a critical task for various industries, including finance, manufacturing, military applications, medicine, mine clearance processes, including time-based analysis of GPR and magnetometer data, and energy in the era of Industry 4.0. The use of recurrent neural networks with LSTM units is an effective approach for predicting long-term dependencies in data. However, the optimal configuration of the long short-term memory (LSTM) architecture remains an open question, in particular, the choice of the number of blocks, the dropout rate, and the use of technical indicators to improve prediction accuracy. This study presents a detailed analysis of the impact of key hyperparameters in LSTM models, including the number of blocks (100-400), the dropout rate (0.1-0.3), and the role of technical indicators such as EMA (Exponential Moving Average) and RSI (Relative Strength Index) in generating accurate forecasts. The obtained results show that EMA_20 has the highest correlation coefficient (0.99) with the closing price, while RSI demonstrates a weaker relationship (0.04-0.05), which emphasizes its secondary role. The training algorithm for the neural network with LSTM blocks was optimized using the Nadam optimizer, which allowed us to determine the most effective combination of hyperparameters for forecasting financial time series. The training data was obtained from the Yahoo Finance (yfinance) library and included historical data on the Google (GOOGL) stock price for the period from 2011 to 2024. The model performance was evaluated using the MSE, RMSE and MAPE metrics, which allowed us to objectively assess the level of forecasting accuracy. The analysis of the obtained results showed that the optimal configuration of the neural network consists of 350 LSTM units, a Dropout level of 0.05, and the Nadam optimizer. This configuration achieved a minimum average absolute percentage error (MAPE) of 1.64%, which is lower than the results obtained in previous studies. The study confirms that increasing the number of LSTM blocks beyond 350 does not improve accuracy and may lead to overfitting.
Time series forecasting is a key task in the era of Industry 4.0, encompassing finance, marketing, energy, medicine, and landmine clearance processes, including time-series analysis of groundpenetrating radar and magnetometer data. One of the most effective tools in this domain is the Long Short-Term Memory (LSTM) recurrent neural network, which has the ability to retain long-term temporal dependencies between data points.
In the context of Intelligent Systems and Technologies in Industry, LSTM models are actively utilized for early failure detection in industrial equipment, forecasting production line loads, identifying anomalous patterns in sensor data, and enhancing the efficiency of logistics management. Conversely, in humanitarian and military fields, LSTM models are applied to mine clearance operations, including time-series analysis of ground-penetrating radar and magnetometer data, forecasting potential landmine displacement due to weather and geological factors, and optimizing 0000-0002-9676-0180 (V. Lytvyn); 0000-0002-7481-8628 (I. Peleshchak); 0000-0001-5271-9883 (Y. Futryk); 0000-00020536-3252 (R. Peleshchak); 0000-0003-2029-7270 (A. Khudyy); 0000-0002-1614-512X (A. Senyk) resource allocation in demining missions. Due to LSTM's ability to retain long-term temporal contexts, these tasks can be solved with greater accuracy and efficiency.
Recurrent neural networks (RNNs) incorporating Long Short-Term Memory (LSTM) units are
extensively employed for time series prediction in modern computational research [
The study in [
In [
The work presented in [
A comparative evaluation of LSTM and Gated Recurrent Unit (GRU) models is conducted in [
In [
Findings from studies [
Several studies [
Research efforts detailed in [
It is noteworthy that Adam, as referenced in multiple studies, utilizes a conventional adaptive weight adjustment approach based on the mean squared gradients and their exponentially smoothed moments. However, Adam does not account for a "lookahead" gradient component. In contrast, the Nadam optimizer incorporates a projected gradient update mechanism, adjusting weights based on anticipated rather than current gradient values. This methodological distinction enables Nadam to enhance the accuracy of LSTM-based time series forecasting. The primary factors influencing forecasting performance include the number of LSTM units, the level of Dropout regularization, and the incorporation of additional technical indicators such as EMA and RSI.
A review of existing literature [
Development of an optimal model architecture and training algorithm with different LSTMblock configurations (100, 200, 300, 400) and trained with the Nadam optimizer. The study aims to identify the most effective Dropout regularization value (0.1, 0.2, 0.3) to prevent overfitting while balancing generalization and accuracy.
Impact of technical indicators such as the Exponential Moving Average (EMA_20) and the Relative Strength Index (RSI) on forecasting accuracy, as these indicators help smooth stochastic fluctuations in data;
Evaluation of model accuracy using metrics such as MSE, RMSE and MAPE.
Thus, the critical issue is designing an optimal LSTM architecture configuration and training algorithm using the Nadam optimizer, selecting the appropriate number of blocks, tuning the Dropout level, and incorporating technical indicators to enhance time series forecasting accuracy.
To achieve this objective, the study focuses on solving the following tasks:
Determining the optimal number of LSTM-blocks in a sequential network to achieve a forecasting error below 1.8%; Optimizing the training process of the LSTM neural network using the Nadam optimizer; Investigating the role of Dropout regularization in improving the predictive performance of LSTM models; Analyzing the impact of EMA and RSI on forecasting accuracy;
Assessing model accuracy using evaluation metrics such as MSE, RMSE and MAPE.
Fig 1. illustrates the architectural design of a recurrent neural network (RNN) [
The processed output from LSTM blocks is subsequently fed into a fully connected (Dense) layer, responsible for generating final prediction outputs. Depending on the specific network configuration, activation functions in LSTM layers may include tanh or sigmoid, while the output Dense layer employs linear or ReLU, depending on the necessity for output scaling.
On Fig. 2 illustrates the internal structure of an individual LSTM unit utilized in this research. (indicating retention), ensuring selective memory update mechanisms cell Сt−1.
f t=σ (W t∗[ ht−1 , xt ]+bf ) ,
( 4 )
where:
f t – represents a vector of values in the range [
W t , bf – denote trainable weight matrices and bias vectors, updated dynamically during the training process.
σ – denotes the sigmoid activation function used to regulate information flow.
2. Input gate — At this stage, the network decides whether to store new information in the current memory state. Specifically, it determines which input values should be used to modify memory. The network utilizes the previous hidden state and the sequence value at the current time step, applying them to a sigmoid function. This process consists of two layers: sigmoid activation layer that determines which values can be updated and tanh activation layer that generates a vector of new candidate values for updating the memory cell it=σ (W i∗[ ht−1 , xt ]+bi) ,
(5) Ct=tan h (W C∗[ ht−1 , xt ]+bC ) ,
(6 ) where: it – represents activation vector that determines memory updates.
Ct – represents vector of new candidate values.
3. Output gate — At this stage, the network processes previously computed and stored information to generate a new hidden state, deciding what will be returned as the output of the current memory cell. First, a sigmoid activation layer determines which part of the current state should be output. Then, the tanh function is applied to compute the candidates for the output state. Finally, all layers’ results are combined, and only the relevant information is returned. ot=σ (W 0∗[ ht−1 , xt ]+b0) ,
(7 ) ht=ot∗tan h (Ct ) ,
(8) where: ot – represents vector of output signal values. ht – represents updated hidden state, which is passed to the next time step.
The LSTM-block structure [
The core idea is that the learning process occurs within a memory-based context. The network forgets, learns, and extracts relevant parts of the information for the next step. However, at the next step, the same process is repeated again. Essentially, this approach attempts to mimic how the human brain learns and retains information through the internal gating mechanisms of LSTM (although this is not necessarily an exact representation, it is an attempt to apply different strategies to improve learning).
The performance of LSTM models was assessed using the following metrics [
Mean Squared Error (MSE) – is an indicator that reflects the average of the squared differences between the actual and predicted values. The use of error squares amplifies the impact of large deviations, making this metric sensitive to significant errors. Formula for calculation MSE: 1 n MSE= ∑ ( yi− ^yi)2 , n i=0
(1)
Root mean squared error (RMSE) – is the square root of MSE, which allows the error to be expressed in the same units as the actual values. This metric provides an intuitive interpretation of forecasting accuracy. The RMSE formula is:
RMSE=√ ∑ ( yi− ^yi)2 , 1 n n i=0 (2)
Mean absolute percentage error (MAPE) – measures the average percentage deviation between actual and predicted values. This metric evaluates forecasting accuracy independently of scale. The MAPE formula is:
MAPE= 1 n y − ^yi|∗100 % . ,
∑| i n i=0 yi
(3)
The computer experiment on time series forecasting was conducted based on a neural network algorithm with LSTM-blocks and the Nadam optimizer, represented in the block diagram in Fig. 3.
The experiment was performed for two different sets of hyperparameters in the LSTM-based neural network.
Case A: First set of parameters:
Dropout: 0.1 Optimizer: Nadam Technical indicators: Exponential Moving Average (EMA) and Relative Strength Index (RSI) were included to enhance the understanding of market trends.
Case B: Second set of parameters:
Number of LSTM-blocks: 350 Dropout: 0.05 Optimizer: Nadam Technical indicators: Exponential Moving Average (EMA) and Relative Strength Index (RSI) were included to enhance the understanding of market trends.
To train the LSTM-based neural network, a dataset was obtained from the publicly available Yahoo Finance source using the yfinance library. The experiment focused on Google (GOOGL) stock market data, forming a dataset that spans January 2011 to January 2024. In total, the analyzed time period includes 4,748 calendar days, while the dataset contains 3,270 records. Each record contains the following characteristics: Open – the opening price of the stock at the beginning of the trading day; High – the highest price recorded during the day; Low – the lowest price at which the stock was traded during the day; Close – the closing price of a stock at the end of the trading session; Adj Close – is the adjusted closing price taking into account corporate events; Volume – the volume of shares that were sold or bought during the day.
The data was retrieved using the yfinance library, which provides a convenient interface for accessing financial data. For this experiment, the “Close” column was selected as it is the most relevant for time series analysis and forecasting.
The model was implemented in Python programming language, using the following libraries: NumPy – for efficient numerical computations. pandas – for handling tabular data and preprocessing. Matplotlib – for graphical visualization of results. keras – a framework for developing and training neural networks. yfinance – a library for retrieving historical financial data.
The dataset was divided into training and test samples in the ratio of 80:20. The Min-Max Scaling method was used to normalize the values. The dataset was prepared in such a way that each forecast was formed on the basis of 60 previous time points. The scaling was performed using the Min-Max Scaler, which contributes to stable model training.
Fig. 4 (a, b) presents the results of the computer experiment on stock price forecasting using an LSTM-based neural network with the first (Case A) and second (Case B) sets of hyperparameters. Fig. 5 (a, b) illustrates the dependency of MAPE metric values on the number of LSTM-blocks for Cases A and B. Fig. 6 (a, b) visualizes the MSE (Mean Squared Error), RMSE (Root Mean Squared Error), and MAPE (Mean Absolute Percentage Error) metrics using histograms.
a. Case A; b. Case B; Figure 4 (a, b): Stock price forecast graphs for the first and second sets of parameters.
Blue line: Training data.
Yellow line: Validation data.
Green line: Predicted values on validation.
a. Case A; b. Case B; Figure 5 (a, b): Dependency of MAPE metric values on the number of LSTM-blocks for cases A and B.
a. Case A – for 275 LSTM-blocks b. Case B – for 325 LSTM-blocks Figure 6 (a, b): Visualization of MSE, RMSE and MAPE metrics using histograms.
Significantly higher MAPE values were observed when using fewer LSTM-blocks (50–100). A more gradual decrease in MAPE with an increasing number of LSTM-blocks indicates the stability of the neural network.
The network morphology in Case A is well-suited for stock price forecasting due to its low error rate and stable predictions. The choice of 300 LSTM-blocks and Dropout 0.1 ensures an optimal balance between model complexity and overfitting prevention. However, despite achieving good accuracy (MAPE = 1.75%), this neural network lags behind Case B in performance.
The network morphology in Case B (350 LSTM-blocks, Dropout 0.05) demonstrated a consistent MAPE value across 180–400 LSTM-blocks, indicating strong generalization capability. In particular, the Case A configuration (300 LSTM-blocks, Dropout 0.1) resulted in minimal MAPE values (~1.64%). Reducing the Dropout rate to 0.05 in Case B helped prevent excessive weight nullification, positively impacting prediction accuracy. This neural network (Case B) outperformed the model from Case A, showing higher efficiency but also greater sensitivity to overfitting with a large number of LSTMblocks (>350). The lowest MAPE value was observed in Case B with 325 LSTM-blocks, indicating that this configuration achieved the highest accuracy and stability. Thus, the best-performing network had 325 LSTM-blocks and a Dropout of 0.05, yielding the lowest MAPE (~1.64%) while minimizing the risk of overfitting.
Histograms (Fig. 6 a, b) provide a visual representation of forecasting accuracy for the neural network models across three key metrics: MSE (Mean Squared Error), RMSE (Root Mean Squared Error) and MAPE (Mean Absolute Percentage Error).
For the first set of hyperparameters (Case A): MSE = 6.57 RMSE = 2.56 MAPE = 1.75%
The experimental results confirmed that using LSTM-based neural networks significantly improves time series forecasting accuracy compared to traditional approaches such as ARIMA and GRU.
In particular, the integration of technical indicators (EMA and RSI) with adaptive optimizers enabled the models to better adapt to complex fluctuations in time series, resulting in more accurate predictions.
Advantages of the proposed LSTM-based neural network in case B configuration: High accuracy: The MAPE of 1.64% confirms the model's ability to provide precise forecasts, even in the presence of stochastic fluctuations in the time series. Robustness to noise: The use of technical indicators allows the model to process time series data more effectively. Flexibility: The ability to adjust the Dropout rate and the number of LSTM-blocks makes the model adaptable to different forecasting tasks and datasets.
During the experiments, the impact of different LSTM-block counts (100, 200, 300, 400) and Dropout levels (0.1, 0.2, 0.3) on the accuracy of Google stock price forecasting was analyzed.
Dropout is a regularization method [18] used to prevent overfitting in a neural network by randomly deactivating (zeroing out) a portion of neurons during training. In our LSTM model, Dropout was applied twice:
After the first LSTM layer (Dropout Layer 1): 10% of neurons were deactivated before passing to the next LSTM layer.
This reduces dependencies between neurons and improves the network’s generalization ability.
Another 10% of neurons were deactivated before passing to the final Dense layer. This helps prevent excessive adaptation of the model to the training data.
Thus, Dropout improves the model’s robustness to input data variability and enhances its generalization on test data.
1. Validation Loss vs. LSTM-blocks (Fig. 7) – This graph illustrates the change in validation loss depending on the number of LSTM-blocks and Dropout values: Increasing the number of LSTM-blocks up to 200 helps minimize validation loss. At 300 LSTM-blocks, the model exhibits the lowest loss values when Dropout 0.2, indicating optimal stability. Dropout 0.3 leads to significant fluctuations, suggesting weaker generalization ability of the model.
2. Training Time vs. LSTM-blocks (Fig. 8) – This graph shows that increasing the number of LSTM-blocks directly impacts the model's training time: There is an almost linear relationship between the number of LSTM-blocks and training time. Dropout has a minimal effect on training time, but minor differences can be observed when the number of blocks exceeds 200.
Fig. 9 presents a 3D-visualization of the stability analysis of LSTM models based on validation loss in the coordinates: number of LSTM-blocks and Dropout.
Optimal Dropout values (0.05–0.1) significantly reduce validation loss during time series forecasting. The LSTM model with 300 blocks demonstrates the best performance in validation forecasting.
The analysis of experimental results confirms that proper tuning of LSTM parameters [19], specifically the number of blocks and Dropout, can significantly improve model accuracy and stability.
3D-visualization of technical parameter influence on forecasting
The architecture of a neural network with LSTM-blocks and its training algorithm were optimized using the Nadam optimizer for time series forecasting and analysis of the impact of hyperparameters and technical indicators on forecasting accuracy.
A block diagram of the neural network algorithm with different numbers of LSTM-blocks and hyperparameter configurations was developed for time series forecasting.
Based on the conducted experiments, the role of the number of LSTM-blocks, Dropout levels, and technical indicators (EMA and RSI) was analyzed: The optimal configuration was achieved with 300–350 LSTM-blocks and Dropout values in the range of 0.05–0.1, minimizing prediction errors. EMA_20 was identified as the key predictor for closing price forecasting, whereas RSI exhibited a weaker correlation, but can be useful for detecting anomalous market movements. Training time increases linearly with the number of LSTM-blocks. The Nadam optimizer ensured stable model training. The EMA_20 trend almost perfectly follows the Close price trend, confirming its efficiency in forecasting. 3D-vizualization of validation loss demonstrated that Dropout = 0.3 is less effective, as it reduces prediction accuracy.
Practical implications of the study: The proposed model can be applied to mine clearance operations, including temporal analysis of georadar and magnetometer data, as well as energy forecasting in the Industry 4.0 era, stock prices, currency exchange rates, sales volume predictions, product demand, and other time series tasks. The study established that using 325 LSTM-blocks is optimal, achieving a minimum forecast error (MAPE = 1.64%), surpassing previous studies where the error exceeded 1.8%. This model can also be adapted for forecasting drone trajectories, meteorological changes, and object recognition based on electromagnetic signals. It was confirmed that applying the Nadam optimizer and low Dropout values (0.05–0.1) ensures training stability and high-speed model learning.
Thus, this study confirms that a properly optimized LSTM architecture, incorporating the optimal number of blocks, Dropout levels, and technical indicators, can significantly enhance the accuracy of time series forecasting. The results obtained open new opportunities for applying LSTM networks in financial analysis, market trend forecasting, and artificial intelligence tasks. Additionally, the proposed architectural and morphological solutions can be directly applied to predicting the trajectories of drones and forecasting the potential movement of mines due to weather and geological factors, as well as optimizing resource planning in demining missions.
The research was carried out with the grant support of the National Research Fund of Ukraine "Methods and means of active and passive recognition of mines based on deep neural networks", project registration number 273/0024 from 1/08/2024 (2023.04/0024). Also, we would like to thank the reviewers for their precise and concise recommendations that improved the presentation of the results obtained.
[18] A Theoretically Grounded Application of Dropout in Recurrent Neural Networks. Advances in Neural Information Processing Systems (NeurIPS), 2016; 29. ttps://doi.org/10.48550/arXiv.1512.05287 [19] Nelson, D. M., Pereira, A. C. M., & de Oliveira, R. A. Stock market's price movement prediction with LSTM neural networks. International Joint Conference on Neural Networks (IJCNN), 2017; 1419-1426. https://doi.org/10.1109/IJCNN.2017.7966019