Principal Bit Analysis: Autoencoding with Schur-Concave Loss
  We consider a linear autoencoder in which the latent variables are quantized,
or corrupted by noise, and the constraint is Schur-concave in the set of latent
variances. Although finding the optimal encoder/decoder pair for this setup is
a nonconvex optimization problem, we show that decomposing the source into its
principal components is optimal. If the constraint is strictly Schur-concave
and the empirical covariance matrix has only simple eigenvalues, then any
optimal encoder/decoder must decompose the source in this way. As one
application, we consider a strictly Schur-concave constraint that estimates the
number of bits needed to represent the latent variables under fixed-rate
encoding, a setup that we call \emph{Principal Bit Analysis (PBA)}. This yields
a practical, general-purpose, fixed-rate compressor that outperforms existing
algorithms. As a second application, we show that a prototypical
autoencoder-based variable-rate compressor is guaranteed to decompose the
source into its principal components.
