Accurate blood glucose (BG) prediction is necessary for daily glucose management of diabetes therapy. As glucose dynamics are often affected by various factors, such as diet, physical exercise, and insulin injection, it is difficult to consider all the relevant information and make a balance between the high-dimensional inputs and learning efficiency for a deep learning network. In this work, a novel multivariate predictor with a multi-scale long shortterm memory (MS-LSTM) network was developed to automatically characterize the high-dimensional temporal dynamics and extract the features of blood glucose fluctuation and temporal trends sufficiently. Meanwhile, a multi-lag structure is designed for multiple variables, which can extract the dependence between different variables and blood glucose fluctuations more effectively. Furthermore, long-term sparse information was encoded and compressed to improve the learning efficiency of this deep learning network. The predictive capability of the proposed method was illustrated through 30-min and 60-min ahead glucose prediction in the OhioT1DM-2 Dataset. The root means square error (RMSE) values of 30-min and 60-min ahead predictions were 19.048 and 32.029, respectively, and the mean absolute error (MAE) values of 30-min and 60-min ahead predictions were 13.503 and 23.833. The results demonstrate the efficiency and prediction accuracy of the offline deep learning network, especially in the case of high-dimensional variables availability.
Diabetes is a chronic disease characterized by the inability to
maintain glucose homeostasis. Healthy pancreas controls the release of
glucagon and insulin through -cells and -cells, respectively, to
maintain normal blood glucose levels [
This paper is organized as follows: section 2 explains the data preprocessing used; section 3 describes the architecture of the MSLSTM network; section 4 illustrates model-free prediction methods in case of missing data; section 5 analyses the experimental results; section 6 summarizes the main contents from this study. 2
The variables selected for prediction included BG value, basal
insulin dosage, bolus insulin dosage, carbohydrate intake, and
timestamp [
The data in OhioT1DM-2 Dataset was collected by multiple devices,
and some of the data was manually recorded by the patient, which
caused the raw data to be asynchronous [
CGM measurements contain noise because of physical interference. Therefore, outlier detection and reconciliation are necessary to remove potential noise. Firstly, a gaussian process regression (GPR) model was trained to detect outliers of CGM measurements. The training dataset of the GPR model was the first 288 points of the training dataset. The input of the GPR model was CGM measurements from time t 30 to t 5, 6 points in total, and its output was mean ( (t)) and variance ( 2(t)) of the CGM prediction at the time t. Then (t) and 2(t) was used to reconcile CGM outlier at the time t as equation(1).
8 (t) > g(t) = < >:g(t) 4:5 2(t) , g(t) < (t) 4:5 2(t) (t) + 4:5 2(t) , g(t) > (t) + 4:5 2(t)
(1) ,others where g(t) is the BG level at time t. 2.3
In the OhioT1DM-2 Dataset, basal insulin dosage and CGM measurements have missing data in some situations. As the basal insulin dosage has daily periodicity, it can be filled by the previous day’s data. Although many methods are applied for missing CGM value filling, the accumulative error will inevitably increase as the number of filling increasing. Therefore, to degrade the accumulative error caused by data filling, the first-order Taylor series extrapolation and historical averages were weighted and summed to fill in the missing CGM values as the number of continuous missing items was less than 12. The respective methods for the missing numbers greater than or equal to 12 will be explained in detail later. It should be noted that the missing CGM values in the training dataset will not be filled to avoid additional noise. 2.4
Data normalization can accelerate deep network training and
improve the accuracy of the model to a certain extent. We used three
methods to normalize the data, and the results show that the model
with coefficient normalization had the best performance. Coefficient
normalization refers to only scale the amplitude of data to maintain
the distribution of the raw data as much as possible [
Specifically, for predicting blood glucose after 30 minutes, the three scales adopted for the blood glucose variable were 1×7, 2×7, and 3×7, which means that all scale levels are 7, and the dilated sampling rate is 1, 2 and 3, respectively. Three lags of the basal variable were 8, 16, and 24, respectively. To ensure the unity of the output dimensions, in the multi-scale hierarchical and multi-lag structure, the number of LSTM states was equal to the minimum scale of blood glucose variable. As shown in Table 3, to sufficiently extract the useful information of various variables, the number of LSTM states in the feature fusion layer was 256. The number of nodes in the fully connected layer later was 256, 64, and 1, respectively, and some dropout layers are added between the fully connected layers to avoid the network overfitting problem. In this section, we will introduce the architecture of the MS-LSTM model and explain how the model is trained and tested. The training data was divided into a training set and a verification set at a ratio of 9:1. The last 10% of the training dataset is closest to the
Blood Glucose
testing dataset in time, and its distribution is most similar to the testing dataset, so it was set apart as the verification set. When training the model, each iteration was evaluated on the verification set. When the model had not obtained better results after 300 consecutive evaluations, the training would be stopped, and the model which performs best on the verification set before would be saved. The training stop strategy that can effectively avoid the problem of overfitting the network is called early stopping. Because the 13th point on the test set needs to be predicted, some training data was added at the beginning of the test set to ensure that the number of prediction points meets the requirements. Besides, for several CGM data after a noticeable amount of continuously missing data, the model was not used for prediction. Instead, two model-free prediction algorithms with adaptive weight prediction and remain prediction were used to predict, respectively. Finally, the predictions were limited in the range of 40 to 400. The flow diagram is shown in Figure 2.
Training batch size: The experiment used mini-batch for weight adjustment, and the batch size of each update weight will affect the accuracy of the model. In the experiment, it was found that the larger batch size could improve the accuracy and accelerate the training process of the model, so the batch size was set to 1024.
Loss function: The experiment compared the negative loglikelihood (NLL) loss function, "-insensitive loss function, mean absolute error (MAE) loss function, and root mean square error (RMSE) loss function. The results displayed that the model trained with the RMSE loss function had the best performance.
Optimizer: This experiment tested the root mean square prop
(RMSProp) optimizer and adaptive moment estimation (Adam) optimizer
[
The experimental environment is Win10 Professional 64-bit operating system, the hardware platform is Intel Core i7 9750H processor, NVIDIA GeForce GTX 1660 Ti graphics processing unit, 16G memory notebook computer, and the development tool is Python 3.6, Keras 2.2.4, TensorFlow-GPU 1.12.0. The code used in the experiment is available on Github. In this hardware and software environment, the average training time for the MS-LSTM model was about 10 minutes.
When the number of the missing CGM data is more than 11, the predictions of the MS-LSTM model for the following several values will cause a significant deviation. Therefore, for these CGM data, adaptive weight prediction and remain prediction are used instead of the model. The adaptive weight prediction algorithm uses short-term maintainability and long-term periodicity of blood glucose levels to make predictions. Specifically, when fewer CGM data are missing, the prediction is close to the last CGM value before the missing data, that is, depending on the short-term maintainability of blood glucose levels. On the contrary, when there are more missing data, the prediction is close to the CGM value at the same time of the previous day, that is, depending on the long-term periodicity of blood glucose levels. The process of adaptive weight prediction can be described by equation (2)-(4).
f gav(t)
Paw = = = nmiss=(nmiss + c) 288+n
X g(t where nmiss is the number of missing data between the current prediction and the last CGM measurement before the missing data. c is a constant not less than 0, and the value in this experiment is set to 68. f is the adaptive weight factor, depend on nmiss and c. T is the blood glucose measurement period, the value in the OhioT1DM-2 Dataset is 5 minutes. n is a positive integer constant not less than 0, and the value in this experiment is set to 1. nback 2 f288 n; 288 n + 1; :::; 288 + ng. g(t) is the BG level at time t. gav(t) is the average value of the CGM data of 2n + 1 points at the same time on the previous day, which represents the long-term periodicity of BG levels. glast(t) is the last CGM value before the missing data, which represents the short-term maintainability of BG levels. Finally, Paw is the adaptive weight prediction value. As shown in Figure 3, the black points in the period from time D to F are the predictions produced by adaptive weight prediction algorithm.
Subject 567 - PH=30 - 22th February,2027
Raw data Model Predictions Adaptive Weight Predictions Remain Prediction
Back Extrapolations 250 225 )L200 d /gm175 ( e sco150 u l dG125 o loB100 75 50
When the first CGM data appears after the missing data, the value would be directly used as the predicted value of the required prediction horizon. So this algorithm is called remain prediction. As shown in the sky blue point in Figure 3, the blood glucose value at time D was the prediction value at time G.
Then, when two CGM values appeared after the missing data, as shown about the BG values at time D and E in Figure 3. Based on these two points, the reverse first-order Taylor series extrapolation was performed. Then the extrapolated data and the average historical data before the missing data were weighted and summed to ensure the smoothness of the filled data. The green points in Figure 3 were the extrapolated backward data, which were used by the MS-LSTM model to predict BG level after time G. 5
The performance of the model was evaluated by the root mean square error (RMSE) and mean absolute error (MAE) between the predictions and the original test data.
RM SE
M AE = = v uu 1
N
X (y^i t N i=1 1 XN jy^i N i=1 yij yi)2 (5) (6) where y^i is the predicted BG value, yi is the target value and N represents the size of the testing dataset. To be noted that, the extrapolated values of BG were removed when evaluating the performance of the model, which guarantees the predictions had the same number as the test data.
According to the preceding steps, the results of four independent experiments are summarized in Table 5, where SD represents the standard deviation. All subjects used the same experimental parameters, but the RMSE of each patient varied from 15 to 22. Among them, the smallest RMSE is 15.871 for patient 596, and the largest RMSE is 21.934 for patient 567. The prediction results are shown in Figure 4-5. It is worth noting that the average RMSE variance of the MS-LSTM model is only 0.061 in 30 minutes prediction horizon, which reflects the excellent robustness of the model.
Subject 596 - PH=30 400 350 )dL300 / g (em250 so lcu200 G lood150 B100 50 400 350 )dL300 / g (em250 so lcu200 G lood150 B100 50 0 0 Raw data Predictions Raw data
Predictions 1500
Time Index 500 1000 2000 2500 3000
The subject 567 has many consecutive spikes, which is the primary source for the prediction error. Besides, another source of prediction
RMSE and MAE values of the MS-LSTM model for 6 subjects. error is the missing data, as shown in the predictions after the missing data in Figure 6. Finally, a slight time delay is observed in the prediction curve, and it is also a problem for most prediction methods.
The CGM measurements contain noise because of physical interference. We used the GPR model to detect and reconcile CGM outliers to the greatest extent. However, only some severe outliers were detected and reconciled because there was no judgment standard for outliers. There are still many outliers in the raw CGM data, which is very unfavorable for the prediction model learning. Therefore, denoising CGM and obtaining high-quality data is very important to improve the performance of the prediction model.
Raw data Predictions 400 350 50 12:00 15:00 18:00 Time (h) 21:00 24:00
In this paper, the MS-LSTM network is developed to adaptively characterize high-dimensional temporal dynamics and extract the longterm and short-term features of glucose fluctuation. Meanwhile, a multi-lag structure is designed for multiple variables, which can extract the dependence between different variables and blood glucose fluctuations more effectively. The long-term sparse temporal data is encoded and compressed to suitable for efficient learning with the model. The mean value of the RMSE for 6 subjects is 19.048, with standard deviation equals to 0.061 in 30-minute PH. Missing data and rapid fluctuations in blood glucose levels are the two main factors that affect the prediction performances of the model. 7