Off-Policy Estimation of Long-Term Average Outcomes with Applications to
  Mobile Health
  Due to the recent advancements in wearables and sensing technology, health
scientists are increasingly developing mobile health (mHealth) interventions.
In mHealth interventions, mobile devices are used to deliver treatment to
individuals as they go about their daily lives. These treatments are generally
designed to impact a near time, proximal outcome such as stress or physical
activity. The mHealth intervention policies, often called just-in-time adaptive
interventions, are decision rules that map an individual's current state (e.g.,
individual's past behaviors as well as current observations of time, location,
social activity, stress and urges to smoke) to a particular treatment at each
of many time points. The vast majority of current mHealth interventions deploy
expert-derived policies. In this paper, we provide an approach for conducting
inference about the performance of one or more such policies using historical
data collected under a possibly different policy. Our measure of performance is
the average of proximal outcomes over a long time period should the particular
mHealth policy be followed. We provide an estimator as well as confidence
intervals. This work is motivated by HeartSteps, an mHealth physical activity
intervention.
