Counterfactual Evaluation of Slate Recommendations with Sequential
  Reward Interactions
  Users of music streaming, video streaming, news recommendation, and
e-commerce services often engage with content in a sequential manner. Providing
and evaluating good sequences of recommendations is therefore a central problem
for these services. Prior reweighting-based counterfactual evaluation methods
either suffer from high variance or make strong independence assumptions about
rewards. We propose a new counterfactual estimator that allows for sequential
interactions in the rewards with lower variance in an asymptotically unbiased
manner. Our method uses graphical assumptions about the causal relationships of
the slate to reweight the rewards in the logging policy in a way that
approximates the expected sum of rewards under the target policy. Extensive
experiments in simulation and on a live recommender system show that our
approach outperforms existing methods in terms of bias and data efficiency for
the sequential track recommendations problem.
