Generalizable Information Theoretic Causal Representation
  It is evidence that representation learning can improve model's performance
over multiple downstream tasks in many real-world scenarios, such as image
classification and recommender systems. Existing learning approaches rely on
establishing the correlation (or its proxy) between features and the downstream
task (labels), which typically results in a representation containing cause,
effect and spurious correlated variables of the label. Its generalizability may
deteriorate because of the unstability of the non-causal parts. In this paper,
we propose to learn causal representation from observational data by
regularizing the learning procedure with mutual information measures according
to our hypothetical causal graph. The optimization involves a counterfactual
loss, based on which we deduce a theoretical guarantee that the
causality-inspired learning is with reduced sample complexity and better
generalization ability. Extensive experiments show that the models trained on
causal representations learned by our approach is robust under adversarial
attacks and distribution shift.
