Enhancing the Interpretability of Deep Models in Heathcare Through
  Attention: Application to Glucose Forecasting for Diabetic People
  The adoption of deep learning in healthcare is hindered by their "black box"
nature. In this paper, we explore the RETAIN architecture for the task of
glusose forecasting for diabetic people. By using a two-level attention
mechanism, the recurrent-neural-network-based RETAIN model is interpretable. We
evaluate the RETAIN model on the type-2 IDIAB and the type-1 OhioT1DM datasets
by comparing its statistical and clinical performances against two deep models
and three models based on decision trees. We show that the RETAIN model offers
a very good compromise between accuracy and interpretability, being almost as
accurate as the LSTM and FCN models while remaining interpretable. We show the
usefulness of its interpretable nature by analyzing the contribution of each
variable to the final prediction. It revealed that signal values older than one
hour are not used by the RETAIN model for the 30-minutes ahead of time
prediction of glucose. Also, we show how the RETAIN model changes its behavior
upon the arrival of an event such as carbohydrate intakes or insulin infusions.
In particular, it showed that the patient's state before the event is
particularily important for the prediction. Overall the RETAIN model, thanks to
its interpretability, seems to be a very promissing model for regression or
classification tasks in healthcare.
