A Reinforcement Learning System to Encourage Physical Activity in
  Diabetes Patients
  Regular physical activity is known to be beneficial to people suffering from
diabetes type 2. Nevertheless, most such people are sedentary. Smartphones
create new possibilities for helping people to adhere to their physical
activity goals, through continuous monitoring and communication, coupled with
personalized feedback.
  We provided 27 sedentary diabetes type 2 patients with a smartphone-based
pedometer and a personal plan for physical activity. Patients were sent SMS
messages to encourage physical activity between once a day and once per week.
Messages were personalized through a Reinforcement Learning (RL) algorithm
which optimized messages to improve each participant's compliance with the
activity regimen. The RL algorithm was compared to a static policy for sending
messages and to weekly reminders.
  Our results show that participants who received messages generated by the RL
algorithm increased the amount of activity and pace of walking, while the
control group patients did not. Patients assigned to the RL algorithm group
experienced a superior reduction in blood glucose levels (HbA1c) compared to
control policies, and longer participation caused greater reductions in blood
glucose levels. The learning algorithm improved gradually in predicting which
messages would lead participants to exercise.
  Our results suggest that a mobile phone application coupled with a learning
algorithm can improve adherence to exercise in diabetic patients. As a learning
algorithm is automated, and delivers personalized messages, it could be used in
large populations of diabetic patients to improve health and glycemic control.
Our results can be expanded to other areas where computer-led health coaching
of humans may have a positive impact.
