Indeterminacy in Latent Variable Models: Characterization and Strong
  Identifiability
  Most modern latent variable and probabilistic generative models, such as the
variational autoencoder (VAE), have certain indeterminacies that are
unresolvable even with an infinite amount of data. Recent applications of such
models have indicated the need for \textit{strongly} identifiable models, in
which an observation corresponds to a unique latent code. Progress has been
made towards reducing model indeterminacies while maintaining flexibility, most
notably by the iVAE (arXiv:1907.04809 [stat.ML]), which excludes many -- but
not all -- indeterminacies. We construct a full theoretical framework for
analyzing the indeterminacies of latent variable models, and characterize them
precisely in terms of properties of the generator functions and the latent
variable prior distributions. To illustrate, we apply the framework to better
understand the structure of recent identifiability results. We then investigate
how we might specify strongly identifiable latent variable models, and
construct two such classes of models. One is a straightforward modification of
iVAE; the other uses ideas from optimal transport and leads to novel models and
connections to recent work.
