The Adaptive Doubly Robust Estimator for Policy Evaluation in Adaptive
  Experiments and a Paradox Concerning Logging Policy
  The doubly robust (DR) estimator, which consists of two nuisance parameters,
the conditional mean outcome and the logging policy (the probability of
choosing an action), is crucial in causal inference. This paper proposes a DR
estimator for dependent samples obtained from adaptive experiments. To obtain
an asymptotically normal semiparametric estimator from dependent samples with
non-Donsker nuisance estimators, we propose adaptive-fitting as a variant of
sample-splitting. We also report an empirical paradox that our proposed DR
estimator tends to show better performances compared to other estimators
utilizing the true logging policy. While a similar phenomenon is known for
estimators with i.i.d. samples, traditional explanations based on asymptotic
efficiency cannot elucidate our case with dependent samples. We confirm this
hypothesis through simulation studies.
