Generalization Bounds and Representation Learning for Estimation of
  Potential Outcomes and Causal Effects
  Practitioners in diverse fields such as healthcare, economics and education
are eager to apply machine learning to improve decision making. The cost and
impracticality of performing experiments and a recent monumental increase in
electronic record keeping has brought attention to the problem of evaluating
decisions based on non-experimental observational data. This is the setting of
this work. In particular, we study estimation of individual-level causal
effects, such as a single patient's response to alternative medication, from
recorded contexts, decisions and outcomes. We give generalization bounds on the
error in estimated effects based on distance measures between groups receiving
different treatments, allowing for sample re-weighting. We provide conditions
under which our bound is tight and show how it relates to results for
unsupervised domain adaptation. Led by our theoretical results, we devise
representation learning algorithms that minimize our bound, by regularizing the
representation's induced treatment group distance, and encourage sharing of
information between treatment groups. We extend these algorithms to
simultaneously learn a weighted representation to further reduce treatment
group distances. Finally, an experimental evaluation on real and synthetic data
shows the value of our proposed representation architecture and regularization
scheme.
