Decision-Focused Learning in Restless Multi-Armed Bandits with
  Application to Maternal and Child Care Domain
  This paper studies restless multi-armed bandit (RMAB) problems with unknown
arm transition dynamics but with known correlated arm features. The goal is to
learn a model to predict transition dynamics given features, where the Whittle
index policy solves the RMAB problems using predicted transitions. However,
prior works often learn the model by maximizing the predictive accuracy instead
of final RMAB solution quality, causing a mismatch between training and
evaluation objectives. To address this shortcoming we propose a novel approach
for decision-focused learning in RMAB that directly trains the predictive model
to maximize the Whittle index solution quality. We present three key
contributions: (i) we establish the differentiability of the Whittle index
policy to support decision-focused learning; (ii) we significantly improve the
scalability of previous decision-focused learning approaches in sequential
problems; (iii) we apply our algorithm to the service call scheduling problem
on a real-world maternal and child health domain. Our algorithm is the first
for decision-focused learning in RMAB that scales to large-scale real-world
problems. \end{abstract}
