Operator-valued formulas for Riemannian Gradient and Hessian and
  families of tractable metrics
  We provide an explicit formula for the Levi-Civita connection and Riemannian
Hessian for a Riemannian manifold that is a quotient of a manifold embedded in
an inner product space with a non-constant metric function. Together with a
classical formula for projection, this allows us to evaluate Riemannian
gradient and Hessian for several families of metrics on classical manifolds,
including a family of metrics on Stiefel manifolds connecting both the constant
and canonical ambient metrics with closed-form geodesics. Using these formulas,
we derive Riemannian optimization frameworks on quotients of Stiefel manifolds,
including flag manifolds, and a new family of complete quotient metrics on the
manifold of positive-semidefinite matrices of fixed rank, considered as a
quotient of a product of Stiefel and positive-definite matrix manifold with
affine-invariant metrics. The method is procedural, and in many instances, the
Riemannian gradient and Hessian formulas could be derived by symbolic calculus.
The method extends the list of potential metrics that could be used in manifold
optimization and machine learning.
