Fast Physical Activity Suggestions: Efficient Hyperparameter Learning in
  Mobile Health
  Users can be supported to adopt healthy behaviors, such as regular physical
activity, via relevant and timely suggestions on their mobile devices.
Recently, reinforcement learning algorithms have been found to be effective for
learning the optimal context under which to provide suggestions. However, these
algorithms are not necessarily designed for the constraints posed by mobile
health (mHealth) settings, that they be efficient, domain-informed and
computationally affordable. We propose an algorithm for providing physical
activity suggestions in mHealth settings. Using domain-science, we formulate a
contextual bandit algorithm which makes use of a linear mixed effects model. We
then introduce a procedure to efficiently perform hyper-parameter updating,
using far less computational resources than competing approaches. Not only is
our approach computationally efficient, it is also easily implemented with
closed form matrix algebraic updates and we show improvements over state of the
art approaches both in speed and accuracy of up to 99% and 56% respectively.
