Online Blood Glucose Prediction Using Autoregressive Moving Average Model with Residual Compensation Network

Abstract

Blood glucose (BG) prediction plays an important role in daily BG control. Accurate prediction of short-term glucose concentration can provide early warning for hyperglycemia and hypoglycemia events. This paper proposed a novel framework that combined an online prediction model with a residual compensation network. The autoregressive moving average (ARMA) model was used for online blood glucose prediction and the neural network was applied for compensation of prediction error. The advantages of this combined framework are: (1) the online ARMA model is efficient and robust to capture time-varying glucose dynamics, (2) the residual compensation network is capable to estimate errors from the online prediction model. The performance of this method was evaluated by the root mean squared error (RMSE) and the mean absolute error (MAE) in the dataset of OhioT1DM.The results were shown in detail that the mean values of the best RMSE of six patients at 30-min and 60-min horizon were 20.03 and 34.89 respectively, and the best MAE at 30-min and 60-min horizon were 14.52 and 24.61. Compared with the ARMA model, the combined predictor with a residual compensation network shows better prediction accuracy. Thus, we concluded that the proposed framework was an available approach for online blood glucose level prediction (BGLP).

INTRODUCTION

Nowadays, daily BG management is a significant challenge for a patient with diabetes. Further improvement of glucose control can be realized through prediction, which allows users to take actions ahead of time to minimize the occurrence of adverse glycemic events [ 3 ]. Thus, accurate blood glucose prediction plays an important role in blood glucose control. However, multiple factors influence glucose variability and lead to different responses between individuals under the same conditions. The prediction of short-term glucose concentration has become an urgent problem for researchers. In the past, various machine learning approaches were proposed to develop datadriven glucose predictive models [ 22 ]. John et al. [ 13 ] used Recurrent Neural Networks that trained in an end-to-end fashion to predict future blood glucose levels through historical blood glucose data. Jaouher et al. [ 2 ] applied an Artificial Neural Networks model to predict future blood glucose levels and hypoglycemic events of Type 1 Diabetes Mellitus (T1DM). The results proved that the model was accurate, adaptive, and encouraging by clinical implementation. Reymann et al. [ 19 ] trained a Support Vector Regression model with an online software simulator. They provided the foundation for the further development of the mobile prediction.

Nevertheless, every prediction algorithm has its own advantages and disadvantages. The ARMA model can be constructed easily by several steps, but they lack the ability to deal with the nonlinear patterns [ 15 ]. Due to the extremely non-stationary characteristic of the time series, the single artificial intelligence models sometimes stuck into the local minimum and fail to achieve satisfactory performance. With the development of equipment, the generation of data flow is continuous. Tracking the time-varying characteristic of the system is crucial. Regarding the non-stationary time series, most scholars adopted one online learning method to model the complex system. The input of the data can adjust the parameters of the model in realtime [ 12 ]. The data of blood glucose is non-stationary, aperiodic, and individuality. Therefore, the use of only one method for BG prediction may give one-sided results [ 14 ]. We need to combine various prediction methods to cover the disadvantages.

In this paper, we proposed a novel framework that combined an online prediction model with a residual compensation network. The ARMA model was used for online blood glucose prediction and the neural network was applied for compensation of prediction error. The advantages of this combined framework are: (1) the online ARMA model is efficient and robust to capture time-varying glucose dynamics, (2) the residual compensation network is capable to estimate errors from the online prediction model. The accuracy of this method was evaluated by short-term glucose prediction in the data set of OhioT1DM.

This paper is structured as following five parts: section I presents a brief literature review that discusses related works on short-term glucose prediction technique; section II presents our method for data preprocessing; section III introduces the principle of the online ARMA model and neural network, as well as the overall framework; section IV discusses the performance of our method on clinical data, and section V concludes the paper. 2

DATA PREPROCESSING

The data used in this paper is provided by the BGLP challenge. OhioT1DM dataset recorded 8-week CGMs data and corresponding daily events from 6 patients with type 1 diabetes, including numbers 540, 544, 552, 567, 584, and 596, respectively. During data collection and transmission, the errors in calibration or measurements may be produced many missing or outlier data points in clinical data. Although, time series models do not consider any physiological factors and only use recent BG data and other inputs that may affect BG levels. The missing data will have a significant effect on the accuracy of the models [ 21 ].

Online models emphasize the real-time input of data streams, hence, the missing data can only be estimated using past data [ 7 ]. Our workflow for dealing with missing data problem is as follows. Based on the CGM data, a time grid with a 5-minute sample period was derived and the missing data were filled with zeros. Firstly, we made a statistical analysis of the size and number of missing data segments through excel software. In both the test set and the training set, there are more than 5% and even 20% missing data. Among them, the loss of blood glucose between 1-100 is relatively common, which may be caused by the replacement of CGM in patients. Secondly, with the statistical results, a backward pushing method or mean value method was implemented for each missing. With the increase of filling times, the cumulative error will inevitably increase.For the training set, the missing CGM values were filled with spline and the historical average at the same point. When the two values are different, the weighted method is used to fill. The test set is processed as follows :(a) the first three positions of the missing segment are filled with extrapolation method;(b) starting from the fourth position of the missing segment, weight the first-order Taylor series extrapolation and average (the historical average at the same point and historical average) to fill;(c) from position 12 of the missing paragraph uses backward induction. Finally, unbroken data would be obtained for prediction. Although many models with multiple inputs (insulin dose, food intake, etc.) can effectively predict the future BG levels. However, the data-collection process of those inputs heavily relies on the subjective inputs provided by the user who wears a CGM device. Since the user may not be professional, the data may be inaccurate and have errors. Due to such limitations, we predicted the future BG level only based on the historical BG data. 3

METHODS AND REALIZATION

In this section, we will introduce the models that are used in the framwork and explain how the proposed framework works for prediciton. 3.1 3.1.1

ARMA model

ARMA model

ARMA, which includes the autoregressive (AR) model and movingaverage (MA) model, is an important method to study the time series [ 17 ]. It is widely used in the prediction of finance and wind power [ 1 ], [ 20 ]. The ARMA could establish linear and nonlinear dynamic models by associating input and output data. And it can be expressed as follows:

p yt = X

q iyt i + X i 1 Where p is the order of the autoregressive part, i is the autoregressive parameter, q is the order of the moving average part, j is the moving average parameter, and t is the error term at time t. In general, the offline parameter determination uses the Least-squares and the online uses Kalman filter. 3.1.2

Online model

The ARMA includes three iterative steps including model identification, parameter estimation, and diagnostic checking. Stationarity is a necessary condition in building an ARMA model which is useful for forecasting. In the identification step, data transformation is often required to make the time series stationary. Meanwhile, the sliding window technique is more robust to the stochastic changes in the data trend and can be applied to smaller datasets [ 25 ]. Hence, the sliding window technique was added to the ARMA model. Discarding old data from the training window can limit the influence of distant past trends during model training and can promote the learning of new trends in the data.

Differencing was applied to it to remove the trend and stabilize the variance because of the trend of the blood glucose data. After that, the sliding window updated the BG data. The method can reduce the training time of the model because the number of training sets is always fixed. As far as we know, determining the order of models is a key to the ARMA model. Akaike’s Information Criterion (AIC) is widely used to optimize the model parameters in those models. AIC is an estimation for the likelihood of a model. However, AIC does not have any indication of the absolute quality. The Bayesian Information Criterion (BIC) is a similar criterion for model selection [ 9 ]. Then, AIC and BIC were used to select an appropriate order in this paper. For the two results, we limit the interval value of the global model order, to conduct experiments to find the optimal model parameters. The last step of model building is the diagnostic checking of model adequacy. If the model is not adequate, a new tentative model should be identified, which is again followed by the steps of parameter estimation and model verification. The ability of the ARMA model in learning small data sets and tracking fast is taken full advantage and can achieve the online update learning. 3.2 3.2.1

Residual compensation network

Neural network

Backpropagation (BP) neural network is a model that can approximate various nonlinearities in the data. It is a kind of multi-layer feedforward neural network trained according to the error propagation algorithm and there are three layers including the input layer, hidden layer, and output layer. In essence, the BP neural network takes the network error square as the objective function and uses the gradient descent method to calculate the minimum value of the objective functio [ 6 ]. Modifying the weight and threshold is the core of the BP neural network. It aims to get the model whose output is consistent with expected results. In this paper, the input layer of the neural network is the predictive value of blood glucose, and the output is the prediction error. The structure is shown in Figure 1.

In Figure 1, n is the number of nodes in the hidden layer, p and q are the number of nodes in the input layer and output layer respectively. The number of hidden layers can be determined according to the empirical formula: n = pp + q + a (2) Where a is the adjustment constant between 1 and 10. The number of input layers is determined by correlation analysis. Then, the best number of hidden layers is determined by the experiment to follow equation (2). It is generally believed that increasing the number of hidden layers can reduce the network error and improve the accuracy, but also complicate the network, thus increasing the network training time and the tendency of overfitting. 3.2.2

Framework of residual compensation

Both the ARMA model and BP neural network have achieved successes in their own linear or nonlinear domains. Neither of them is suitable for all circumstances. The statistical methods have their linear limitations, which means that they cannot simulate the real-time series with nonlinear mode well [ 5 ]. On the other hand, a single BP neural network is not enough to capture the time patterns contained in highly complex time series. In the training process of the BP neural network, there may be problems of the model following error and uncertainty, resulting in the generation of overfitting or underfitting model [ 11 ]. A hybrid methodology can be a good strategy for practical use [ 24 ] , [ 4 ]. It combines different models to capture different aspects of the underlying patterns. Ji et al. [ 10 ] used the ARMA model to predict linear components of the time series, and the TDNN model to predict nonlinear components. Results showed that the model had the advantages of both two methods and the prediction accuracy of the model was improved. However, only the optimal combinations of different models can obtain the best hybrid models, the framework of the hybrid models becomes very important.

In this paper, we proposed a novel framework that combined an online ARMA model with a residual compensation network (RCNARMA) to predict BG. The blood glucose data which belongs to chaotic time series contains linear and nonlinear components [ 8 ]. Due to the randomness and volatility of BG, the ARMA model inevitably produces large errors in the prediction of nonlinear nonstationary time-series data, which has a certain tendency and periodicity [ 23 ]. The BP neural network has good data error tolerance, but it is insufficient for linear prediction. Since the ARMA model cannot capture the nonlinear structure of the BG data. The residuals of the linear model will contain information about the nonlinearity. The BP neural network is valid for satisfying the prediction effect of most non-linear properties. Hence, the BP neural network was applied to predict residuals. The framework aimed to reduce the uncertainty of model selection and improve the model forecasting performance by dealing with both linear and nonlinear patterns in time series. The flow diagram of the RCN-ARMA is shown in Figure 2.

The specific prediction process of RCN-ARMA is as follows: Step 1. The sliding window updates the input for the ARMA model. AIC and BIC are used to confirm the order of ARMA. Then predict the blood glucose by the online ARMA model.

Step 2. Compared to the predicted value with the raw data, the residual time series, which is used to the compensation network, can be constructed.

Step 3. The correlation analysis of the predicted values and residual time series is carried out to determine the input of the residual compensation network [ 16 ]. According to the results of the correlation analysis, the range of input variables may be different from 6 patients.

Step 4. As an important supplement to model prediction, a common three-layer neural network is applied to predict the residual. The neural network predicts the errors in the future based on a series of errors in the past and can overcome the influence of various uncertainties changes on system stability.

Step 5. Analysis of blood glucose predictions and residual time series in statistically. The output display value range of the CGM is [40,400], and the error is basically within the range of [-50,50]. For this reason, some rules are employed to correct discrete data points appropriately in the research.

Step 6. Combine the results of the two-step prediction and get the final After the five steps, we have got the prediction results of the BG and the error. Combine the results of the two-step prediction by the direct sum method and get the final BG prediction. 4 4.1

RESULTS AND DISCUSSION

Evaluating indicator

For model evaluation, general and commonly used evaluation methods are sensitivity, specific, root mean square error (RMSE), and mean absolute error (MAE) [ 18 ]. In this paper, two widely used evaluation indexes were applied to compare the prediction capacity. The error indexes define as below:

RM SE

M AE = = v uu 1

N

X (y^i t N i=1 1 XN jy^i N i=1 yij yi)2 (3) (4) Where: y^i represents the predicted value, yi represents the real value and N represents the size of the data set. Two rules were applied in the evaluation: 1) as long as the corresponding timestamp had the raw data, the RMSE and MAE indexes of the test set would be calculated; 2) If there was a null value in the input model data, it meant that the data was given insufficiently, and the value at this time would not be recorded. We only use the first and the code used during the experiment is available on Github. In this paper, the predictions of the model were recorded from the thirteenth point of the test set. And the results were recorded as two decimal places rounded. 4.2

Results

In this section, the results and analysis of the proposed framework are presented. The online AR, BP, and ARMA models were used for 30-min ahead predictions. The mean values of the RMSE and MAE for six patients are shown in Table 1. Then the RCN-ARMA was used to 30-min and 60-min ahead predictions. The experiments were conducted on patients with different inputs by establishing an online ARMA model and a residual compensation network (RCNARMA). The optimal value of the sliding window was selected by the experimental method and keeps the same in two networks. Due to the heterogeneity of the patients themselves, the selected parameters had some differences. The 30-min ahead predictions of ARMA and RCN-ARMA for 540, 567 patients are graphically shown in Figure 3 and Figure 4. Table 2 shows the RMSE and MAE of the different contributors for 30-min and 60-min ahead predictions. Based on the results in table2, mean RMSE and MAE of 30-min and 60-min ahead predictions respectively with the online ARMA and RCN-ARMA are shown in Table 3. The above tables contain the results of three cases, and the reliability of the conclusions is enhanced through a comparison of multiple cases.

Table 1 shows that different models have different prediction effects on blood glucose prediction. The ARMA model is better than the other two models in prediction. The reason is that the online ARMA model has an advantage in tracking real-time changes of data.And the AR model which does not contain the moving average model (MA) is a special form of ARMA model. There is a big difference in MAE between the two. Therefore, we choose online ARMA as the base model. Figure 3 and Figure 4 clearly illustrated that (a) the predicted value of ARMA has obvious lag on the whole, which is one of the main reasons affecting the prediction effect of the model; (b) the addition of the error compensation model improves the hysteresis of the predicted value of the model; (c) the mixed prediction results show sharp fluctuations and a certain amount of peak data that are negative effects of adding compensation.

As can be seen from Table 2 and Table 3: (a) for different patients, the model prediction effect is different and reflects the specificity of blood glucose data; (b) prediction ability of the model got worse with the increase of the prediction step. This is a major issue that needs to be addressed urgently; (c) through the correlation analysis of predicted value residuals, it implies that a significant correlation relationship exists for the multi-step ahead forecast error series of ARMA. Thus, it is very useful for the error forecast models to select effective input variables in this multi-step ahead forecasting model; (d) compared with the online ARMA model, the evaluating indicator of RCN-ARMA all decreased, especially for 60-min ahead predictions; (e) from the drop value, the change of two different step size evaluation indexes gradually increases. The overall effect decreases with the increase of prediction step size for both models. The improvements of the proposed combined framework compared with a certain individual model increase with increasing prediction steps for the continuous multi-step ahead forecasting.

Discussion

To further compare the performance difference between models, the effectiveness of the proposed model is demonstrated by the promoting percentage of between models. The data collected from 6 patients is used as our case study. The simulation results demonstrate that the proposed forecasting framework improves the short-term blood glucose forecasting accuracy significantly compared with the reference models. The residual compensation network can timely predict the errors to supplement the missing nonlinearity information of the ARMA model. The proposed framework not only retains the advantage of the ARMA model for fast-tracking a small amount of data but also covers the shortage of nonlinear learning which mainly affects the overall improvement of the results. For the neural networks, the advantage of the framework can be reflected by its ability of nonlinear prediction. It proves that the framework can better capture the nonlinear and linear characteristics of the time series. Compared with using a single algorithm, this framework is more comprehensive. At present, both time series and machine learning algorithms have their disadvantages. People have been studying the corresponding matching algorithm to solve the disadvantage of the algorithm. This framework of models can be promoted to an individual model by fixing known flaws using a complementary model. 5

CONCLUSION

In this study, a new framework for blood glucose prediction based on the online ARMA model with residual compensation network was proposed. The online ARMA model was applied for predicting dynamic changes of blood glucose in real-time, and the residual model was used to track the errors of the online model. Prediction results of 6 patients, the RCN-ARMA had much higher prediction accuracy than the ARMA model. The proposed framework improved the ability of ARMA model prediction and proposed a better short-term prediction performance. Because the accuracy of the ARMA model in blood glucose prediction is improved, the application of the ARMA model in the artificial pancreas (AP) system will have better safety and stability. From the time series prediction results, the framework is also applicable to the integration of other prediction models to achieve clinical applications. The aim was to cover the missing useful prediction information caused by the shortcomings of the single model. Future work, we will further select an appropriate evolutionary algorithm to optimize the model parameters. 6

FUNDING REFERENCES

References

This research was supported by National Natural Science Foundation of China (No.61973067 and No . 61903071 ).

[1] L. Arisena , A. and Noviyanti and S. Achmad Zanbar , ' Portfolio return using black-litterman single view model with arma-garch and treynor black model' , in Journal of Physics: Conference Series , volume 974 , Surabaya , Indonesia, ( 2018 ).

[2] J. Ben Ali , T. Hamdi , N. Fnaiech , V. Di Costanzo , F. Fnaiech , and J. -M. Ginoux , ' Continuous blood glucose level prediction of type 1 diabetes based on artificial neural network', Biocybernetics and Biomedical Engineering, 38 , 828 - 40 , ( 2018 ).

[3] R.H. Botwey , E. Daskalaki, P. Diem , and S.G. Mougiakakou , ' Multimodel data fusion to improve an early warning system for hypo/hyperglycemic events' , in 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) , pp. 4843 - 6 , Piscataway, NJ, USA, ( 2014 ).

[4] P. S.G. de Mattos Neto , G.D.C. Cavalcanti , and F. Madeiro , ' Nonlinear combination method of forecasters applied to pm time series' , Pattern Recognition Letters , 95 , 65 - 72 , ( 2017 ).

[5] J. F. L. de Oliveira , L. D. S. Pacifico , P. S. G. de Mattos Neto , E. F. S. Barreiros , C. M.O. Rodrigues , and A. T. A. Filho , ' A hybrid optimized error correction system for time series forecasting' , Applied Soft Computing Journal , 87 , ( 2020 ).

[6] Sh. Ding and Q.H. Wu , ' A matlab-based study on approximation performances of improved algorithms of typical bp neural networks' , in Applied Mechanics and Materials , pp. 1353 - 1356 , ( 2013 ).

[7] E. I. Georga , V. C. Protopappas , D. Polyzos , and D. I. Fotiadis , ' Online prediction of glucose concentration in type 1 diabetes using extreme learning machines' , in Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society , EMBS, pp. 3262 - 3265 , ( 2015 ).

[8] T. Hamdi , V. Di Costanzo , F. Fnaiech , E. Moreau, R. Naeck , and J.M. Ginoux , ' Glycemic evolution of type 1 diabetic patients is a chaotic phenomenon' , in IECON Proceedings (Industrial Electronics Conference) , pp. 5177 - 5181 , Florence, Italy, ( 2016 ).

[9] R.J. Hyndman and G. Athanasopoulos , Forecasting: Principles and Practice, 3rd edition , OTexts: Melbourne, Australia, 2019 .

[10] W. Ji and K. C. Chee , ' Prediction of hourly solar radiation using a novel hybrid model of arma and tdnn' , Solar Energy , 85 , 808 - 817 , ( 2011 ).

[11] X.Y. Kong , Li Ch ., Ch .Sh. Wang , Y.S. Zhang , and J. Zhang, ' Shortterm electrical load forecasting based on error correction using dynamic mode decomposition' , Applied Energy , 261 , ( 2020 ).

[12] M. Langkvist , L. Karlsson , and A. Loutfi , ' A review of unsupervised feature learning and deep learning for time-series modeling' , Pattern Recognition Letters , 42 , 11 - 24 , ( 2014 ).

[13] K. Li , J. Daniels , L Chengyuan , P. Herrero , and P. Georgiou , ' Convolutional recurrent neural networks for glucose prediction' , IEEE Journal of Biomedical and Health Informatics , 24 , 603 - 613 , ( 2020 ).

[14] K.Z. Li , Ch.Y. Liu, T.Y. Zhu , P. Herrero , and P. Georgiou , ' Glunet: A deep learning framework for accurate glucose forecasting' , IEEE Journal of Biomedical and Health Informatics , 24 , 414 - 423 , ( 2020 ).

[15] Y.F. Li , H.P. Wu , and H. Liu, 'Multi-step wind speed forecasting using ewt decomposition, lstm principal computing, relm subordinate computing and iewt reconstruction' , Energy Conversion and Management , 167 , 203 - 219 , ( 2018 ).

[16] Zh . T. Liang , J. Liang , Ch.F. Wang , X.M. Dong , and X.F. Miao , ' Shortterm wind power combined forecasting based on error forecast correction' , Energy Conversion and Management , 119 , 215 - 226 , ( 2016 ).

[17] N. S. Nalawade and M.M. Pawar , ' Forecasting telecommunications data with autoregressive integrated moving average models' , in 2015 2nd International Conference on Recent Advances in Engineering and Computational Sciences, RAECS 2015 , Chandigarh, India, ( 2015 ).

[18] S. E. Noujaim , D. Horwitz , M. Sharma , and J. Marhoul , ' Accuracy requirements for a hypoglycemia detector: An analytical model to evaluate the effects of bias, precision, and rate of glucose change' , Journal of Diabetes Science & Technology, 1(5) , 652 - 668 , ( 2007 ).

[19] M.P. Reymann , E. Dorschky , B.H. Groh , C. Martindale , P. Blank , and B.M. Eskofier , ' Blood glucose level prediction based on support vector regression using mobile platforms' , in 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) , pp. 2990 - 3 , Piscataway, NJ, USA, ( 2016 ).

[20] H. Sharadga , S. Hajimirza , and R. S. Balog , ' Time series forecasting of solar power generation for large-scale photovoltaic plants' , Renewable Energy , 150 , 797 - 807 , ( 2020 ).

[21] C.F. Tsai and Y.C. Chen , ' The optimal combination of feature selection and data discretization: An empirical study' , Information Sciences , 505 , 282 - 293 , ( 2019 ).

[22] A. Z. Woldaregay , E. Arsand , S. Walderhaug , D. Albers , L. Mamykina , T. Botsis , and G. Hartvigsen, ' Data-driven modeling and prediction of blood glucose dynamics: Machine learning applications in type 1 diabetes' , Artificial Intelligence in Medicine , 98 , 109 - 134 , ( 2019 ).

[23] J. Yang , L. Li , Y.M. Shi , and X.L. Xie , ' An arima model with adaptive orders for predicting blood glucose concentrations and hypoglycemia' , IEEE Journal of Biomedical and Health Informatics , 23 , 1251 - 1260 , ( 2019 ).

[24] G.P. Zhang , ' Time series forecasting using a hybrid arima and neural network model' , Neurocomputing , 50 , 159 - 75 , ( 2003 ).

[25] L.Y. Zhang , J.B. Zhao , and W. Li , ' Online and unsupervised anomaly detection for streaming data using an array of sliding windows and pdds' , IEEE Transactions on Cybernetics , 1- 6 , ( 2019 ).