Coupling-based Invertible Neural Networks Are Universal Diffeomorphism
  Approximators
  Invertible neural networks based on coupling flows (CF-INNs) have various
machine learning applications such as image synthesis and representation
learning. However, their desirable characteristics such as analytic
invertibility come at the cost of restricting the functional forms. This poses
a question on their representation power: are CF-INNs universal approximators
for invertible functions? Without a universality, there could be a well-behaved
invertible transformation that the CF-INN can never approximate, hence it would
render the model class unreliable. We answer this question by showing a
convenient criterion: a CF-INN is universal if its layers contain affine
coupling and invertible linear functions as special cases. As its corollary, we
can affirmatively resolve a previously unsolved problem: whether normalizing
flow models based on affine coupling can be universal distributional
approximators. In the course of proving the universality, we prove a general
theorem to show the equivalence of the universality for certain diffeomorphism
classes, a theoretical insight that is of interest by itself.
