On Pitfalls of Identifiability in Unsupervised Learning. A Note on:
  "Desiderata for Representation Learning: A Causal Perspective"
  Model identifiability is a desirable property in the context of unsupervised
representation learning. In absence thereof, different models may be
observationally indistinguishable while yielding representations that are
nontrivially related to one another, thus making the recovery of a ground truth
generative model fundamentally impossible, as often shown through suitably
constructed counterexamples. In this note, we discuss one such construction,
illustrating a potential failure case of an identifiability result presented in
"Desiderata for Representation Learning: A Causal Perspective" by Wang & Jordan
(2021). The construction is based on the theory of nonlinear independent
component analysis. We comment on implications of this and other
counterexamples for identifiable representation learning.
