Variational Optimization on Lie Groups, with Examples of Leading
  (Generalized) Eigenvalue Problems
  The article considers smooth optimization of functions on Lie groups. By
generalizing NAG variational principle in vector space (Wibisono et al., 2016)
to Lie groups, continuous Lie-NAG dynamics which are guaranteed to converge to
local optimum are obtained. They correspond to momentum versions of gradient
flow on Lie groups. A particular case of $\mathsf{SO}(n)$ is then studied in
details, with objective functions corresponding to leading Generalized
EigenValue problems: the Lie-NAG dynamics are first made explicit in
coordinates, and then discretized in structure preserving fashions, resulting
in optimization algorithms with faithful energy behavior (due to conformal
symplecticity) and exactly remaining on the Lie group. Stochastic gradient
versions are also investigated. Numerical experiments on both synthetic data
and practical problem (LDA for MNIST) demonstrate the effectiveness of the
proposed methods as optimization algorithms ($not$ as a classification method).
