Lie Group Auto-Encoder
  In this paper, we propose an auto-encoder based generative neural network
model whose encoder compresses the inputs into vectors in the tangent space of
a special Lie group manifold: upper triangular positive definite affine
transform matrices (UTDATs). UTDATs are representations of Gaussian
distributions and can straightforwardly generate Gaussian distributed samples.
Therefore, the encoder is trained together with a decoder (generator) which
takes Gaussian distributed latent vectors as input. Compared with related
generative models such as variational auto-encoder, the proposed model
incorporates the information on geometric properties of Gaussian distributions.
As a special case, we derive an exponential mapping layer for diagonal Gaussian
UTDATs which eliminates matrix exponential operator compared with general
exponential mapping in Lie group theory. Moreover, we derive an intrinsic loss
for UTDAT Lie group which can be calculated as l-2 loss in the tangent space.
Furthermore, inspired by the Lie group theory, we propose to use the Lie
algebra vectors rather than the raw parameters (e.g. mean) of Gaussian
distributions as compressed representations of original inputs. Experimental
results verity the effectiveness of the proposed new generative model and the
benefits gained from the Lie group structural information of UTDATs.
