Bandit Algorithms for Precision Medicine
  The Oxford English Dictionary defines precision medicine as "medical care
designed to optimize efficiency or therapeutic benefit for particular groups of
patients, especially by using genetic or molecular profiling." It is not an
entirely new idea: physicians from ancient times have recognized that medical
treatment needs to consider individual variations in patient characteristics.
However, the modern precision medicine movement has been enabled by a
confluence of events: scientific advances in fields such as genetics and
pharmacology, technological advances in mobile devices and wearable sensors,
and methodological advances in computing and data sciences.
  This chapter is about bandit algorithms: an area of data science of special
relevance to precision medicine. With their roots in the seminal work of
Bellman, Robbins, Lai and others, bandit algorithms have come to occupy a
central place in modern data science ( Lattimore and Szepesvari, 2020). Bandit
algorithms can be used in any situation where treatment decisions need to be
made to optimize some health outcome. Since precision medicine focuses on the
use of patient characteristics to guide treatment, contextual bandit algorithms
are especially useful since they are designed to take such information into
account. The role of bandit algorithms in areas of precision medicine such as
mobile health and digital phenotyping has been reviewed before (Tewari and
Murphy, 2017; Rabbi et al., 2019). Since these reviews were published, bandit
algorithms have continued to find uses in mobile health and several new topics
have emerged in the research on bandit algorithms. This chapter is written for
quantitative researchers in fields such as statistics, machine learning, and
operations research who might be interested in knowing more about the
algorithmic and mathematical details of bandit algorithms that have been used
in mobile health.
