Neural Network Forecasting Management in the Supply Chain Method for Inventory

Abstract

  Determining the optimal level of inventory comes down to the timeliness of the procurement and replenishment procedures, which ensure the minimum total costs associated with procurement and storage. The problem of insufficient prediction accuracy for inventory management arising in supply chains is considered. A neural network prediction model based on a Time-Delay Restricted Boltzmann Machine with unit delays cascades in the output and input layers is proposed. During the structural identification of this model, the neurons count in the hidden layer was calculated, and the parametric identification was performed based on the CUDA parallel processing technology. This improves the prediction efficiency by increasing the prediction accuracy and decreasing the computational complexity. Software has been developed by the Matlab package that realizes the offered method. The created software is used to implement the prediction in the supply chain management problem.

Keywords

•  1  prediction accuracy
• supply chain management problem
• neural network prediction model
• Restricted Boltzmann Machine
• stochastic learning

1. Introduction 

 initial data about the features may be missing;  source data can be noisy, incomplete or highly correlated;  systems analysis with a big nonlinearity degree is possible;  rapid model create;  high adaptability;  systems analysis with a big number of features is possible;  full list of all possible models is not demanded;  systems analysis with heterogeneous features is possible.

Given the advantages above, the paper will use a artificial neural network prediction method.

The goal of the work is to create a artificial neural network prediction method in the supply chain to improve prediction accuracy. To achieve the aim, the next tasks were solved:  analyze existing prediction methods;  propose a artificial neural network prediction model;  choose a criterion for evaluating the artificial neural network prediction model quality;  propose a method for calculating parameters of the artificial neural network prediction model;  conduct a numerical study.

2. Formal problem statement 

Let the learning set S  {(x , d )} ,  1, P be given for the prediction.

The problem of enhancement the prediction accuracy for the Time-Delay Restricted Boltzmann Machine (TDRBM) model g(x,W ) , where x – input vector, W – parameters vector, is showed as the problem of searching for this model such a parameters vector W * that meets the criterion F  1 P

 (g(x ,W *)  d )2  min .

P  1

3. Literature review 

The often used prediction artificial neural networks are:  Jordon's neural network (JNN) [13, 14]. It is a recurrent two-layer network and is founded on multilayer perceptron (MLP). JNN is recommended for prediction stationary short-term signals;  Elman's neural network (ENN) [15, 16]. It is a recurrent two-layer network and is founded on MLP. SRN is recommended for prediction stationary short-term signals;  recurrent multilayer perceptron (RMLP) [17, 18]. It is a recurrent multilayer network and is founded on MLP. RMLP is recommended for prediction stationary short-term signals;  nonlinear autoregressive model (NAR) [19, 20]. It is a non-recurrent two-layer network and is founded on MLP. NAR is recommended for prediction stationary short-term signals;  nonlinear autoregressive moving average model (NARMA) [21, 22]. It is a recurrent twolayer network and is founded of MLP. NARMA is recommended for prediction stationary shortterm signals;  bidirectional recurrent neural network (BRNN) [23, 24]. It is a recurrent two-layer network and is founded on two Elman networks. BRNN is recommended for prediction non-stationary long-term signals.

Table 1 shows the comparative features of prediction artificial neural networks. As follows from Table 1, most neural networks have one or more of the following disadvantages:  have a high probability of hitting a local extremum;  do not have the ability to train in batch mode;  have no feedback.

Due to this, it is relevant to create a neural network that will eliminate the indicated disadvantages.

Table 1 

Comparative features of prediction artificial neural networks   Figure 1: Block diagram of a prediction model TDRBM for visible input neurons (TDRBM type first) 

Output neurons

Input neurons

Output neurons

Input

neurons … t-1

Visible

neurons … … t-1

4. Block diagram of the neural network prediction model 

Unlike the traditional Restricted Boltzmann Machine (RBM) [25, 26], unit delay cascades are used for neurons in the visible layer. TDRBM type first has unit delay cascade in the visible input layer. TDRBM type seconf has unit delay cascade in the visible input layers and visible output layers.

5. Neural network prediction models 

5.1.  Components of the prediction model TDRBM 

The components of the TDRBM model are stochastic neurons, the state of which is described based on the Bernoulli distribution in the form

1, with probability Pj x j   .

0, with probability 1  Pj The probability of transition of the j th stochastic neuron to state 1 is defined as , where E j – is the TDRBM energy increment when the state of the j th stochastic neuron changes from 0 to 1.

5.2.  Prediction model TDRBM type first 

Initial value assignment of the time delay input neurons state 2. Initial value assignment of the input and output neurons state 3. where wtiinjh – weights between the input layer and the hidden layer, wouth – weights between the output layer and the hidden layer, ij bhj – bias of the hidden layer neurons, M in – the unit delays count for the input layer, N in – the neurons count in the input layer, N h – the neurons count in the hidden layer, N out – the neurons count in the output layer.

4. Calculation of the hidden neurons state x hj ( )  1, Pj  U ( 0,1 ) 0, Pj  U ( 0,1 ) where U ( 0,1 ) – standard uniform distribution on a line [0,1] . , j 1, N h where bojut – bias of the output layer neurons. , j 1, N out

5.3.  Prediction model based on TDRBM type second 

  1 .

Initial value assignment time moment of the time delay input and output neurons state xin (  t)  0 , t 1, M in , xout (  t)  0 , t 1, M out .

xin ( )  xin , xout ( )  0 . 2. Initial value assignment of the input and output neurons state 3. where wtiinjh –weights between the input layer and the hidden layer, wtoijuth – weights between the output layer and the hidden layer, bhj – bias of the hidden layer neurons, M in – the unit delays count for the input layer, M out – the unit delays count for the output layer, N in – the neurons count in the input layer,  M out N out N h  1  exp  bojut    wtoijut out xiout (  t)   w0oiujt h xih ( )  t 1 i1 i1  , j 1, N h , , j 1, N out The outcome is (x1out ( ),..., xNouotut ( )) . 6. Criterion for the artificial neural network prediction model quality 

For learning the TDRBM, a model adequacy criterion was selection, which means the selection of such parameters W  {wtiinjh , wtoijuth , wtiinjin , wtoijut out} , which get a mean squared error (MSE) minimum:

F  1 P N out

  (xouit  di )2  min .  P  1 i1 W ( 1 ) 

The learning of the TDRBM model is taking into account the criterion ( 1 ). 7. Methods for calculating the neural network prediction model parameters  7.1.  Principles  of  calculating  the  neural  network  prediction  model  parameters 

The determination of the TDRBM parameters is perform on the found of the CD-1 method, which speeds up stochastic learning with a teacher, since instead of stabilizing the state of neurons, it performs only one step of adjusting their state. TDRBM operates in two phases: positive and negative.

For TDRBM type first, in the positive phase, visible input neurons are fixed at times from t  M in to t and visible output neurons at time t , and the TDRBM performs one step of tuning hidden neurons. In the negative phase, the visible input neurons are first fixed at times from t  M in to t 1 , the hidden neurons are trained in the positive phase, and the TDRBM performs one step of tuning the visible input and output neurons at time t . After that, the visible input neurons are fixed at times from t 1 to t  M in , and the visible input and output neurons trained in the negative phase at time t , and the TDRBM performs one step of tuning the hidden neurons.

For TDRBM type second, in the positive phase visible input neurons are fixed at times t  M in to t , visible output neurons at times from t  M out to t , and the TDRBM performs one step of tuning hidden neurons. In the negative phase, the visible input neurons are first fixed at times from t  M in to t 1 , the visible output neurons at times from t  M out to t 1 , the hidden neurons are trained in the positive phase, and the TDRBM performs one step of tuning the visible input and output neurons at time t . After that, visible input neurons are recorded at times from t 1 to t  M in , visible output neurons at times from t 1 to t  M out , and visible input and output neurons trained in the negative phase at time t , and TDRBM performs one step of tuning hidden neurons. 7.2.  Method  for  calculating  the  prediction  model  parameters  found  on 

TDRBM type first 

Learning iteration number n  1, initial value assignment by uniform distribution means on the interval ( 0,1 ) or [-0.5, 0.5] bias biin (n) , i 1, N in , biout (n) , i 1, N out , b hj (n) , j 1, N h , and weights wtiinjh (n) , t  0, M in , i 1, N in , j 1, N h , wouth (n) , i 1, N out , j 1, N h , wtiinjin (n) , ij wouth (n)  wojiuth (n) , where M in is the unit delays count for input neurons.

ij 2.

A learning set{(xin , xout ) | xin {0,1}Nin , xout {0,1}Nout } ,  1, P is defined, where xin –  th  learning input vector, xout –  th learning output vector, P is the learning set capacity.

3. Initial value assignment time moment Initial value assignment of the time delay input neurons state

4. Initial value assignment of the input and output neurons state xin ( )  xin , xout ( )  xout .

Calculation of the hidden neurons transition probability to state 1 Pj  

M in N in

N out

Calculation of the hidden neurons state 1, Pj  U ( 0,1 ) xh ( )   j 0, Pj  U ( 0,1 )

8. Initial value assignment time moment Initial value assignment of the time delay input neurons state

Negative phase (steps 8-16) 9. Initial value assignment of the of hidden, input and output neurons state 10. Calculation of the output neurons transition probability to state 1

xin (  t)  0 , x2in (  t)  0 , t 1, M in .

xin ( )  x1in ( ) , xout ( )  x1out ( ) , xh ( )  x1h ( ) .

Saving the neurons state in the positive phase, i.e. x1in ( )  xin ( ) , x1out ( )  xout ( ) , x1h ( )  xh ( ) . If   P , then     1 , go to 4.

1 1 1 11. Calculation of the of output neurons state 12. Calculation of the input neurons transition probability to state 1

1, xojut ( )  

Pj  U ( 0,1 ) 0, Pj  U ( 0,1 ) 13. Calculation of the input neurons state 15. Calculation of the hidden neurons state

1, xi  xj (xi,xj)   0, xi  xj . 21. Tuning of weights between the output layer and the hidden layer founded on Boltzmann's rule 1, P U( 0,1 )

j xijn()   0, Pj U( 0,1 ) 16. Saving the neurons state in the negative phase, i.e. x2in()  xin(), x2out()  xout(), x2h()  xh() . If   P , then   1, go to 9. 17. Tuning of bias input layer founded on Boltzmann's rule 18. Tuning of bias output layer founded on Boltzmann's rule biin(n)  biin(n) 1 P x1iin() x2iin() , i1,Nin .

1 P   P 1 P 1 

 biout(n)  biout(n) 1 P x1iout() x2iout(), i1,Nout .

1 P   P 1 P 1   19. Tuning of bias hidden layer neurons founded on Boltzmann's rule bhj(n)  bhj(n)  1 P x1hj()   x2hj() , j1,Nh .

1 P   P 1 P 1   20. Tuning of weights between the input layer and the hidden layer founded on Boltzmann's rule wtiinjh(n)  wtiinjh(n) (tij tij), t0,Min , i1,Nin , j1,Nh , tij  (x1iin( t),x1hj()), tij  (x2iin( t),x2hj()), 14. Calculation of the hidden neurons transition probability to state 1 23. If the neurons states in the output layer of the positive and negative phases differ slightly, i.e.

 | x1iout()  x2iout()|  , then n  n 1, go to 2.

  P 1 i1 

The method for calculating the prediction model parameter based on TDRBM type first is shown in Figure 3.

1 P P 1

P 1 P 1 1 P P 1 P 1 P 1 22. Tuning of weights between input layers founded on Boltzmann's rule wouth(n)  wiojuth(n)(ij ij) , i1,Nout , j1,Nh ,

ij ij  1 P

(x1iout(),x1hj()), ij  1 P(x2iout(),x2hj()).

wtiinjin(n)  wtiinjin(n)(tij tij) , t1,Min , i, j1,Nin , tij  (x1iin( t),x1ijn()) , tij  1 P(x2iin( t),x2ijn()) , 1 P Figure 3: Flowchart of the method for calculating the prediction model parameters found on TDRBM  type first  7.3. 

Method  for  calculating  the  prediction  model  parameters  found  on 

TDRBM type second 

Learning iteration number n  1, initial value assignment by uniform distribution means on the interval ( 0,1 ) or [-0.5, 0.5] bias biin (n) , i 1, N in , bout (n) , i 1, N out , bh (n) , j 1, N h , and i j weights wtiinjh (n) , t  0, M in , i 1, N in , j 1, N h , wtoijuth (n) , t  0, M out , i 1, N out , j 1, N h , wtiinjin (n) , delays count for input neurons, M out is the unit delays count for output neurons.

A learning set {(xin , xout ) | xin {0,1}Nin , xout {0,1}N out } is defined,  1, P , where xin –  th learning input vector, xout –  th learning output vector, P is the learning set capacity.

Initial value assignment time moment Initial value assignment of the time delay input and output neurons state

Positive phase (steps 3-7)

xin (  t)  0 , x1in (  t)  0 , t 1, M in ,

Pj  Initial value assignment of the input and output neurons state

xin ( )  xin , xout ( )  xout .

Saving the neurons state in the positive phase, i.e. x1in ( )  xin ( ) , x1out ( )  xout ( ) , x1h ( )  xh ( ) . If   P , then     1 , go to 4.

Negative phase (steps 8-16) Initial value assignment time moment

 =1.

Initial value assignment of the time delay input and output neurons state

xin (  t)  0 , x2in (  t)  0 , t 1, M in , xout (  t)  0 , x2out (  t)  0 , t 1, M out .

Initial value assignment of the of hidden, input and output neurons state

xin ( )  x1in ( ) , xout ( )  x1out ( ) , xh ( )  x1h ( ) .

Pj 

Pj  10. Calculation of the output neurons transition probability to state 1

1  M out N out N h  1  exp  bojut (n)    wtoijut out (n)xiout (  t)   w0oiujt h (n)xih ( ) 

 t 1 i1 i1  11. Calculation of the of output neurons state xojut ( )  1, Pj  U ( 0,1 ) , j 1, N out .

0, Pj  U ( 0,1 ) 12. Calculation of the input neurons transition probability to state 1 1 x2h()  xh() . If   P , then   1, go to 9. 17. Tuning of bias input layer founded on Boltzmann's rule biin(n)  biin(n) P1 P1x1iin() 1 Px2iin() , i1,Nin .

 P 1  18. Tuning of bias output layer founded on Boltzmann's rule biout(n)  biout(n) P1 P1x1iout() 1 P x2iout(), i1,Nout .

 P 1  19. Tuning of bias hidden layer neurons founded on Boltzmann's rule bhj(n)  bhj(n)  P1 P1 x1hj()  1 P x2hj() , j1,Nh .

 P 1  20. Tuning of weights between the input layer and the hidden layer founded on Boltzmann's rule 24. If the neurons states in the output layer of the positive and negative phases differ slightly, i.e. 1 P Nout| x1iout()  x2iout()|  , then n  n 1, go to 2.

 P 1 i1 

The method for calculating the prediction model parameters found on TDRBM type second is shown in Figure 4.

8. Experiments and results 

A numerical study of the proposed method for determining the parameter values was carried out using the CUDA technology of parallel processing of information in the Matlab package, wherein the number of threads in the block was 1024, the summation was performed based on the parallel reduction algorithm.

21. Tuning of weights between the output layer and the hidden layer founded on Boltzmann's rule wtiinjh(n)  wtiinjh(n) (tij tij), t0,Min , i1,Nin , j1,Nh , tij  1 P(x1iin( t),x1hj()), tij  1 P(x2iin( t),x2hj()),

P 1 P 1

1, xi  xj (xi,xj)  

0, xi  xj . wtoijuth(n)  wiojuth(n)(ij ij) , t0,M out , i1,Nout , j1,Nh , ij  P1 P1(x1iout(),x1hj()), ij  1 P(x2iout(),x2hj()).

P 1 wtiinjin(n)  wtiinjin(n)(tij tij) , t1,Min , i, j1,Nin , tij  1 P(x1iin( t),x1ijn()) , tij  1 P(x2iin( t),x2ijn()) .

P 1 P 1 wtoijutout(n)  wtoijutout(n)(tij tij), t1,M out , i, j1,Nout , tij  1 P (x1iout( t),x1ojut()), tij  1 P (x2iout( t),x2ojut()).

P 1 P 1 22. Tuning of weights between input layers founded on Boltzmann's rule 23. Tuning of weights between output layers founded on Boltzmann's rule

To determine the prediction model structure TDRBM with 16 input neurons (the bits count of the integer predicted metric), i.e. defining the number of hidden neurons, a experiments count were performed, the results of which are showed in Figure 5.

2 1,8 1,6 1,4 1,2 SE 1 M 0,8 0,6 0,4 0,2 0 2 4 6 8 10 12 16 18 20 22 24

26 Figure 5: The mean squared error dependence on the hidden neurons number 

9. Conclusion 

As source data to calculate the artificial neural network prediction model parameters, a values sample of the logistics company "Ekol Ukraine" economic activities was used. The dataset capacity for the "goods sold" indicator was 1800 (in five years). The dataset was spilt into three parts - valid data (20%), training data (60%), test data (20%). The learning was consistent and occurred over 1000 epochs.

The criterion for selection the artificial neural network model structure of the was the prediction minimum mean squared error (MSE) (Table 2). According to Figure 5, with an increase in the hidden neurons count the error value decreases. For the prediction, it is enough to choose 16 hidden neurons, because with a further increase in their count the change in the MSE value is small.

Table 2 

Comparative features of prediction artificial neural networks  

RMLP 

According to Table 2, recurrent neural networks with deterministic neurons (JNN, ENN(SRN), NARMA, BRNN) are more efficient than non-recurrent neural networks with deterministic neurons (RMLP, NAR); neural networks with stochastic neurons (TDRBM) are more efficient than neural networks with deterministic neurons (RMLP, JNN, ENN(SRN), NAR, NARMA, BRNN), and TDRBM type second has the highest prediction accuracy.

1. To solve the problem of enhancement the prediction accuracy in the supply chain, the modern prediction methods were researched. These researches have shown that today the use of neural networks is the most effective. 2. To improve the efficiency of the prediction, a artificial neural network RBM was modified by the unit delays cascades in the input and output layers. In the experiments, its model structure was determined. 3. A method was offered for calculating the parameters of the offered artificial neural network prediction model based on the CUDA technology of parallel processing of information. This made it possible to ensure accuracy of the prediction and high speed. 4. The offered approach can be used in different intelligent prediction systems. For example, in supply chain inventory management systems such as MRP and Lean Production, where prediction plays an important role. 10.

References 

[6] E. Fedorov, T. Utkina, О. Nechyporenko, Forecast method for natural language constructions based on a modified gated recursive block, in: CEUR Workshop Proceedings, vol. 2604, 2020, pp. 199-214. [7] R. T. Baillie, G. Kapetanios, F. Papailias, Modified information criteria and selection of long memory time series models, in: Computational Statistics and Data Analysis, volume 76, 2014, pp. 116–131. doi: 10.1016/j.csda.2013.04.012. [8] A. Wilinski, Time series modelling and forecasting based on a Markov chain with changing transition matrices, in: Expert Systems with Applications, volume 133, 2019, pp. 163–172. doi: 10.1016/j.eswa.2019.04.067. [9] P. Bidyuk, T. Prosyankina-Zharova, O. Terentiev, Modelling nonlinear nonstationary processes in macroeconomy and finances, in: Advances in Computer Science for Engineering and Education, volume 754, Springer, 2019, pp. 735–745. doi: 10.1007/978-3-319-91008-6_72. [10] B. Choubin, G. Zehtabian, A. Azareh, E. Rafiei-Sardooi, F. Sajedi-Hosseini, Ö. Kişi, Precipitation forecasting using classification and regression trees (CART) model: a comparative study of different approaches, Environ Earth Sciences 77, 314 (2018). doi: 10.1007/s12665-0187498-z. [11] L. Lyubchyk, E. Bodyansky, A. Rivtis, Adaptive harmonic components detection and forecasting in wave non-periodic time series using neural networks, in: Proceedings of the ISCDMCI'2002, Evpatoria, 2002, pp. 433-435. [12] S. N. Sivanandam, S. Sumathi, S. N. Deepa, Introduction to Neural Networks using Matlab 6.0,

The McGraw-Hill Comp., Inc., New Delhi, 2006. [13] E. C. Carcano, P. Bartolini, M. Muselli and L. Piroddi, Jordan recurrent neural network versus ihacres in modelling daily streamflows, Journal of Hydrology 362 (2008) 291-307. [14] H. Hikawa and Y. Araga, Study on gesture recognition system using posture classifier and Jordan recurrent neural network, in: Proceedings of the International Joint Conference on Neural Networks, 2011, pp. 405-412. doi: 10.1109/IJCNN.2011.6033250. [15] Z. Zhang, Z. Tang, C. Vairappan, A novel learning method for Elman neural network using local search, in: Neural Information Processing – Letters and Reviews, vol. 11, 2007, pp. 181–188. [16] A. Wysocki and M. Ławryńczuk, Predictive control of a multivariable neutralisation process using Elman neural networks, in: Advances in Intelligent Systems and Computing, Springer: Heidelberg, 2015, pp. 335-344. doi: 10.1007/978-3-319-15796-234. [17] S. Haykin, Neural networks and Learning Machines, Upper Saddle River, New Jersey: Pearson

Education, Inc., 2009. [18] K.-L. Du, K. M. S. Swamy, Neural Networks and Statistical Learning, Springer-Verlag, London, 2014. [19] J. A. Pucheta, C. M. Rodríguez Rivero, M. R. Herrera, C. A. Salas, H. D. Patiño and B. R.

Kuchen, A feed-forward neural networks-based nonlinear autoregressive model for forecasting time series, Computación y Sistemas 14 ( 4 ) (2011) 423-435. [20] G. P. Zhang, Time series forecasting using a hybrid ARIMA and neural network model,

Neurocomputing 50 (2003) 159–175. doi: 10.1016/S0925-2312(01)00702-0. [21] W.K. Wong, M. Xia, W. C. Chu, Adaptive neural network model for time-series forecasting, European Journal of Operational Research 207( 2 ) (2010) 807-816. doi: 10.1016/j.ejor.2010.05.022. [22] H. Jaeger, H. Haass, Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication, Science 304 (2004) 78–80. doi: 10.1126/science.1091277. [23] M. Sundermeyer, T. Alkhouli, J. Wuebker, H. Ney, Translation modeling with bidirectional recurrent neural networks, in: Proceedings of the Conference on Empirical Methods on Natural Language Processing, 2014, pp. 14-25. [24] M. Berglund, T. Raiko, M. Honkala, L. Kärkkäinen, A. Vetek, J. Karhunen, Bidirectional Recurrent Neural Networks as Generative Models, in: Proceedings of the 28th International Conference on Neural Information Processing Systems, 2015, pp. 856−864. [25] G. E. Hinton, A Practical Guide to Training Restricted Boltzmann Machines, Technical Report

UTML TR 2010–003, University of Toronto, 2010. [26] A. Fischer, C. Igel, Training Restricted Boltzmann Machines: An Introduction, Pattern Recognition 47 (2014) 25-39. doi: 10.1016/j.patcog.2013.05.025.

References

[1] J. F. Cox , J.G. Schleher , Theory of Constraints Handbook, New York, NY, McGraw-Hill , 2010 .

[2] S. Smerichevska et al, Cluster Policy of Innovative Development of the National Economy: Integration and Infrastructure Aspects: monograph , S. Smerichevska (Eds.), Wydawnictwo naukowe WSPIA , 2020 .

[3] E. M. Goldratt , My saga to improve production, Selected Readings in Constraints Management, Falls Church, VA: APICS ( 1996 ) 43 - 48 .

[4] E. M. Goldratt , Production: The TOC Way (Revised Edition) including CD-ROM Simulator and Workbook, Revised edition , Great Barrington, MA: North River Press, 2003 .

[5] T. Neskorodieva , E. Fedorov , I. Izonin , Forecast method for audit data analysis by modified liquid state machine , in: CEUR Workshop Proceedings , 2020 , volume 2631 , pp. 145 - 158 .