Dimension Estimation Using Autoencoders
  Dimension Estimation (DE) and Dimension Reduction (DR) are two closely
related topics, but with quite different goals. In DE, one attempts to estimate
the intrinsic dimensionality or number of latent variables in a set of
measurements of a random vector. However, in DR, one attempts to project a
random vector, either linearly or non-linearly, to a lower dimensional space
that preserves the information contained in the original higher dimensional
space. Of course, these two ideas are quite closely linked since, for example,
doing DR to a dimension smaller than suggested by DE will likely lead to
information loss. Accordingly, in this paper we will focus on a particular
class of deep neural networks called autoencoders which are used extensively
for DR but are less well studied for DE. We show that several important
questions arise when using autoencoders for DE, above and beyond those that
arise for more classic DR/DE techniques such as Principal Component Analysis.
We address autoencoder architectural choices and regularization techniques that
allow one to transform autoencoder latent layer representations into estimates
of intrinsic dimension.
