Confident Off-Policy Evaluation and Selection through Self-Normalized
  Importance Weighting
  We consider off-policy evaluation in the contextual bandit setting for the
purpose of obtaining a robust off-policy selection strategy, where the
selection strategy is evaluated based on the value of the chosen policy in a
set of proposal (target) policies. We propose a new method to compute a lower
bound on the value of an arbitrary target policy given some logged data in
contextual bandits for a desired coverage. The lower bound is built around the
so-called Self-normalized Importance Weighting (SN) estimator. It combines the
use of a semi-empirical Efron-Stein tail inequality to control the
concentration and a new multiplicative (rather than additive) control of the
bias. The new approach is evaluated on a number of synthetic and real datasets
and is found to be superior to its main competitors, both in terms of tightness
of the confidence intervals and the quality of the policies chosen.
