Information Theory in Density Destructors
  Density destructors are differentiable and invertible transforms that map
multivariate PDFs of arbitrary structure (low entropy) into non-structured PDFs
(maximum entropy). Multivariate Gaussianization and multivariate equalization
are specific examples of this family, which break down the complexity of the
original PDF through a set of elementary transforms that progressively remove
the structure of the data. We demonstrate how this property of density
destructive flows is connected to classical information theory, and how density
destructors can be used to get more accurate estimates of information theoretic
quantities. Experiments with total correlation and mutual information
inmultivariate sets illustrate the ability of density destructors compared to
competing methods. These results suggest that information theoretic measures
may be an alternative optimization criteria when learning density destructive
flows.
