Risk Variance Penalization
  The key of the out-of-distribution (OOD) generalization is to generalize
invariance from training domains to target domains. The variance risk
extrapolation (V-REx) is a practical OOD method, which depends on a
domain-level regularization but lacks theoretical verifications about its
motivation and utility. This article provides theoretical insights into V-REx
by studying a variance-based regularizer. We propose Risk Variance Penalization
(RVP), which slightly changes the regularization of V-REx but addresses the
theory concerns about V-REx. We provide theoretical explanations and a
theory-inspired tuning scheme for the regularization parameter of RVP. Our
results point out that RVP discovers a robust predictor. Finally, we
experimentally show that the proposed regularizer can find an invariant
predictor under certain conditions.
