Latent Variables on Spheres for Autoencoders in High Dimensions
  Variational Auto-Encoder (VAE) has been widely applied as a fundamental
generative model in machine learning. For complex samples like imagery objects
or scenes, however, VAE suffers from the dimensional dilemma between
reconstruction precision that needs high-dimensional latent codes and
probabilistic inference that favors a low-dimensional latent space. By virtue
of high-dimensional geometry, we propose a very simple algorithm, called
Spherical Auto-Encoder (SAE), completely different from existing VAEs to
address the issue. SAE is in essence the vanilla autoencoder with spherical
normalization on the latent space. We analyze the unique characteristics of
random variables on spheres in high dimensions and argue that random variables
on spheres are agnostic to various prior distributions and data modes when the
dimension is sufficiently high. Therefore, SAE can harness a high-dimensional
latent space to improve the inference precision of latent codes while maintain
the property of stochastic sampling from priors. The experiments on sampling
and inference validate our theoretical analysis and the superiority of SAE.
