Towards IID representation learning and its application on biomedical
  data
  Due to the heterogeneity of real-world data, the widely accepted independent
and identically distributed (IID) assumption has been criticized in recent
studies on causality. In this paper, we argue that instead of being a
questionable assumption, IID is a fundamental task-relevant property that needs
to be learned. Consider $k$ independent random vectors $\mathsf{X}^{i = 1,
\ldots, k}$, we elaborate on how a variety of different causal questions can be
reformulated to learning a task-relevant function $\phi$ that induces IID among
$\mathsf{Z}^i := \phi \circ \mathsf{X}^i$, which we term IID representation
learning.
  For proof of concept, we examine the IID representation learning on
Out-of-Distribution (OOD) generalization tasks. Concretely, by utilizing the
representation obtained via the learned function that induces IID, we conduct
prediction of molecular characteristics (molecular prediction) on two
biomedical datasets with real-world distribution shifts introduced by a)
preanalytical variation and b) sampling protocol. To enable reproducibility and
for comparison to the state-of-the-art (SOTA) methods, this is done by
following the OOD benchmarking guidelines recommended from WILDS. Compared to
the SOTA baselines supported in WILDS, the results confirm the superior
performance of IID representation learning on OOD tasks. The code is publicly
accessible via https://github.com/CTPLab/IID_representation_learning.
