People with type 1 diabetes must constantly monitor their blood glucose levels and take actions to keep them from getting either too high or too low. Having a snack will raise blood glucose levels; however, the amount of carbohydrates that should be consumed to reach a target level depends on the recent history of blood glucose levels, meals, boluses, and the basal rate of insulin. Conversely, to lower the blood glucose level, one can administer a bolus of insulin; however, determining the right amount of insulin in the bolus can be cognitively demanding, as it depends on similar contextual factors. In this paper, we show that a generic neural architecture previously used for blood glucose prediction in a what-if scenario can be converted to make either carbohydrate or bolus recommendations. Initial experimental evaluations on the task of predicting carbohydrate amounts necessary to reach a target blood glucose level demonstrate the feasibility and potential of this general approach.
Type 1 diabetes is a disease in which the pancreas fails to produce
insulin, which is required for blood sugar to be absorbed into cells.
Without it, that blood sugar remains in the bloodstream, leading to
high blood glucose levels (BGLs). In order to manage type 1
diabetes, insulin must be administered via an external source, such as
injections or an insulin pump. People with type 1 diabetes also need
to monitor their BGLs closely throughout the day by testing the blood
acquired through fingersticks and/or by using a continuous glucose
monitoring (CGM) system. If the BGL gets too high (hyperglycemia)
or too low (hypoglycemia), the individual responds by eating, taking
insulin, or taking some other action to help get their BGL back to
within a healthy range. An issue with this, however, is that the person
with diabetes must react to their BGL, whereas, ideally, they would
be able to proactively control their BGL. There has been much work
in the area of BGL prediction in the past ([
The broad goal of the research presented in this paper is to
essentially reverse the blood glucose prediction problem, and instead
predict how many carbohydrates an individual should eat or how much
insulin to administer with a bolus in order to get their BGL to the
desired target. We have previously introduced in [
The work by Mougiakakou and Nikita [
The data-driven architecture proposed in this paper is generic in the sense that it can be trained to make recommendations about any variable that can impact BG levels, in particular carbohydrates and insulin. The task of making carbohydrate recommendations is potentially useful in scenarios where patients want to prevent hypoglycemia well in advance, or where a person is interested in achieving a relatively higher target BGL in preparation for an exercise event that is expected to lower it.
As a first step, in this paper we approach the problem of making carbohydrate recommendations. The rest of this paper is organized in the following way: Section 2 provides a more detailed description of the problem. Section 3 describes the model as well as the baselines used to compare against. Section 4 describes the dataset that is used and some of the features of the data. Section 5 discusses some of the training techniques and methods used as well as the results of the experiments that motivated the use of these techniques. Section 6 contains the conclusion and some plans for future work. 2
We assume that blood glucose levels are measured at 5 minute intervals through a CGM system. We also assume that discrete deliveries of insulin (boluses) and continuous infusions of insulin (basal rates) are recorded. Subjects provide the timing of meals and estimates of the amount of carbohydrates associated with each meal. Given the data available up to the present time t, the problem can formally be defined as predicting the number of grams of carbohydrates (number of carbs) Ctm in a meal that is to be consumed at time tm 2 [t; t + ) such that the person’s BGL reaches a specified target value BGt+ at time t+ in the future. Without loss of generality, in this paper we set the prediction horizon = 30 and 60 minutes. We define three carbohydrate prediction scenarios, depending on whether events such as boluses or other meals happen inside the prediction window [t; t+ ): 1. Scenario S1 assumes that there are no events in the prediction window [t; t + ). Training a model for this scenario can be difficult due to the scarcity of corresponding training examples, as meals are typically preceded by boluses. The example shown in Figure 1 would be in this scenario if the orange and red outlined meals and boluses were not present. 2. Scenario S2 subsumes scenario S1 by allowing events before the meal, i.e. in the time window [t; tm]. The example that is shown in Figure 1 would be a scenario S2 example if the bolus outlined in red were not present, and would correspond to answering the following what-if question: how many carbs should be consumed at time tm to achieve the target BGt+ , if the meal were to be preceded by another meal and a bolus. 3. Scenario S3 is the most general and allows events to happen during the entire prediction window [t; t + ). The example in Figure 1 is a scenario S3 example but not a scenario S1 or scenario S2 example because of the presence of the orange and red outlined meal and bolus.
We train and evaluate carbohydrate recommendation models for each
scenario, using data acquired from 6 subjects with type 1 diabetes [
Given training data containing meals with their corresponding timestamps and carbohydrates, we define the following baselines: 1. Global average: The average number of carbs over all of the meals in the subject’s training data, , are computed and used as the estimate for all future meals, irrespective of context. This is a fairly simple baseline, as it predicts the same value for every example. 2. ToD average: In this Time-of-Day (ToD) dependent baseline, an average number of carbs is computed for each of the following five time windows during a day: 12am-6am: 1 = early breakfast/late snacks. 6am-10am: 2 = breakfast. 10am-2pm: 3 = lunch. 2pm-6pm: 4 = dinner.
6pm-12am: 5 = late dinner/post-dinner snacks.
The average for each ToD interval is calculated over all of the meals appearing in the corresponding time frame in the subject’s training data. At test time, to predict the number of carbs for a meal to be consumed at time tm, we first determine the ToD interval that contains tm and output the corresponding ToD average. Given sufficient historical data, the ToD baseline is expected to perform well for individuals who tend to eat very consistently and have regular diets. However, it is expected to perform poorly on individuals who have a lot of variation in their diets.
While simple to compute and use at test time, the two baselines are
likely to give suboptimal performance, as their predictions ignore the
history of BG values, insulin (boluses and basal rates), and meals, all
of which could significantly modulate the effect a future meal might
have on the BGL. To exploit this information, we propose the general
neural network architecture shown in Figure 1. The first component
in the architecture is a recurrent neural network (RNN) instantiated
using Long Short-Term Memory (LSTM) cells [
The entire architecture is trained to minimize the mean squared error between the actual carbohydrates Ctm recorded in the training data and the estimated value C^tm computed by the output node of the FCN module. Each LSTM uses vectors of size 100 for the states and gates, whereas the FCN is built with 5 hidden layers, each consisting of 200 ReLU neurons, and one linear output node. 4
The data used for the model was collected from 6 subjects with type 1
diabetes [
Since the total number of available examples is directly related to the number of meals, it is useful to know how many meals each subject had. This is shown in Table 1, together with the average number of carbs per meal (Avg), and the corresponding standard deviation (StdDev). Most subjects have a similar average number of carbohydrates in their meals, with the exception of 570 who has a significantly larger number of carbs per meal on average, and more importantly, a much higher standard deviation than the other subjects.
A meal event occurring at time tm may give rise to multiple
examples, depending on the position of tm in the interval [t; t + ). When
= 30 minutes, an example is created for every possible position
of tm within [t; t + ). However, when = 60 minutes, an
example is created for every position of tm within [t; t + 30], to ensure
that there are at least 30 minutes between the meal and the prediction
horizon. Table 2 below shows the resulting number of examples for
= 30 and 60 minutes, in each of the three scenarios. Note that
there are fewer examples in scenarios S1 and S2 when = 60 vs. 30
minutes, despite there being more scenario S3 examples. This can be
explained by the scenarios S1 and S2 criteria being even more
difficult to meet when = 60 minutes, i.e. there cannot be any event
within [t; t + 60) for S1, or any event within [tm; t + 60) for S2.
The Adam [
Since the overall number of examples available in the dataset is low, the performance was improved by first pretraining a generic model on the combined data from all 6 subjects. Then, for each subject, a new model is initialized with the weights of the generic model, and then fine-tuned on the subject’s training data. For each subject, five models were trained with different seedings of the random number generators. We also experimented with fine-tuning models on the union of the training and validation data instead of just the training data. When this combined data is used, the average carb values used in the baselines are recalculated over the union of the training and validation data for each subject. 5.1
The metrics used to evaluate the performance of the models are the root mean squared error (RMSE) and the mean absolute error (MAE), which is less sensitive to large errors. At the end of the training process, there are five fine-tuned models for each subject. The average RMSE and MAE of the five models are reported, as well as the RMSE and MAE of the best model. The model that is considered the ”best” is the one that had the lowest MAE on the validation data. The results of the five models for each subject are also averaged across all subjects to obtain one overall RMSE and one overall MAE value for the average model and the best model scores. The baselines are treated much the same, as their RMSE and MAE values are averaged across all subjects to give an RMSE and an MAE score for each baseline.
Table 3 compares the validation results achieved in scenario S3 by models with and without pretraining for = 30 minutes. This experiment clearly shows the benefit of pretraining the models: both the RMSE and MAE are noticeably lower for the pretrained models. As a result, pretraining is always used as part of the training process for both values of .
Table 4 compares models that were fine-tuned on training and validation data with models fine-tuned solely on the training data, in scenario S3. The results show that the extra examples provided by the validation data proved helpful in improving performance. It is interesting to note that using the combined training-validation data only slightly helped the baselines, but helped the LSTM-based models by a noticeable margin. Overall, the LSTM-based models (Average or Best) had the best RMSE and MAE performance across all three scenarios, with the exception of the RMSE scores for scenario S1. Compared to the other two scenarios, the LSTM models and the baselines have a lower performance in S1. The decline in performance is even more apparent for the LSTM models, which cannot beat the time-dependent baseline in terms of RMSE for both the 30 minute and 60 minute prediction horizons. This can be explained by the limited number of examples for scenario S1: since there are so few testing examples in this scenario per subject, one bad prediction can hurt the results significantly, more so for the RMSE than the MAE. Furthermore, the trained models tend to make very similar predictions for all examples stemming from a specific meal, meaning that if the model made a bad prediction for one test example, it likely made a series of similarly bad predictions.
To alleviate the scarcity of training examples in scenario S1, models trained on S3 examples, which are the most plentiful and subsume S1, were evaluated separately on test examples from S1. This gives an indication on whether any transfer learning is taking place. Table 6 shows the results of this transfer learning experiment, indicating that training on the additional examples from scenario S3 helps improve performance on scenario S1 to the level that now the LSTM-based models outperform both baselines. We introduced a generic neural architecture, composed of two chained LSTMs and a fully connected network, with the purpose of training data-driven models for making recommendations with respect to any type of quantitative events that may impact BG levels, in particular carbohydrate amounts and bolus insulin dosages. Experimental evaluations on the task of carbohydrate recommendations within a 30 or 60 minute prediction window demonstrate the feasibility and potential of the proposed architecture, as well as its ability to benefit from pre-training and transfer learning. Future plans include evaluating carbohydrate recommendations within larger prediction windows, as well as training the architecture for bolus recommendations.
This work was supported by grant 1R21EB022356 from the National Institutes of Health (NIH). Conversations with Josep Vehi helped shape the research directions presented herein. The contributions of physician collaborators Frank Schwartz, MD, and Amber Healy, DO, are gratefully acknowledged. We would also like to thank the anonymous people with type 1 diabetes who provided their blood glucose, insulin, and meal data.