Estimating and Improving Dynamic Treatment Regimes With a Time-Varying
  Instrumental Variable
  Estimating dynamic treatment regimes (DTRs) from retrospective observational
data is challenging as some degree of unmeasured confounding is often expected.
In this work, we develop a framework of estimating properly defined "optimal"
DTRs with a time-varying instrumental variable (IV) when unmeasured covariates
confound the treatment and outcome, rendering the potential outcome
distributions only partially identified. We derive a novel Bellman equation
under partial identification, use it to define a generic class of estimands
(termed IV-optimal DTRs), and study the associated estimation problem. We then
extend the IV-optimality framework to tackle the policy improvement problem,
delivering IV-improved DTRs that are guaranteed to perform no worse and
potentially better than a pre-specified baseline DTR. Importantly, our
IV-improvement framework opens up the possibility of strictly improving upon
DTRs that are optimal under the no unmeasured confounding assumption (NUCA). We
demonstrate via extensive simulations the superior performance of IV-optimal
and IV-improved DTRs over the DTRs that are optimal only under the NUCA. In a
real data example, we embed retrospective observational registry data into a
natural, two-stage experiment with noncompliance using a time-varying IV and
estimate useful IV-optimal DTRs that assign mothers to high-level or low-level
neonatal intensive care units based on their prognostic variables.
