Basal Glucose Control in Type 1 Diabetes using Deep Reinforcement
  Learning: An In Silico Validation
  People with Type 1 diabetes (T1D) require regular exogenous infusion of
insulin to maintain their blood glucose concentration in a therapeutically
adequate target range. Although the artificial pancreas and continuous glucose
monitoring have been proven to be effective in achieving closed-loop control,
significant challenges still remain due to the high complexity of glucose
dynamics and limitations in the technology. In this work, we propose a novel
deep reinforcement learning model for single-hormone (insulin) and dual-hormone
(insulin and glucagon) delivery. In particular, the delivery strategies are
developed by double Q-learning with dilated recurrent neural networks. For
designing and testing purposes, the FDA-accepted UVA/Padova Type 1 simulator
was employed. First, we performed long-term generalized training to obtain a
population model. Then, this model was personalized with a small data-set of
subject-specific data. In silico results show that the single and dual-hormone
delivery strategies achieve good glucose control when compared to a standard
basal-bolus therapy with low-glucose insulin suspension. Specifically, in the
adult cohort (n=10), percentage time in target range [70, 180] mg/dL improved
from 77.6% to 80.9% with single-hormone control, and to $85.6\%$ with
dual-hormone control. In the adolescent cohort (n=10), percentage time in
target range improved from 55.5% to 65.9% with single-hormone control, and to
78.8% with dual-hormone control. In all scenarios, a significant decrease in
hypoglycemia was observed. These results show that the use of deep
reinforcement learning is a viable approach for closed-loop glucose control in
T1D.
