Mirrorless Mirror Descent: A Natural Derivation of Mirror Descent
  We present a primal only derivation of Mirror Descent as a "partial"
discretization of gradient flow on a Riemannian manifold where the metric
tensor is the Hessian of the Mirror Descent potential. We contrast this
discretization to Natural Gradient Descent, which is obtained by a "full"
forward Euler discretization. This view helps shed light on the relationship
between the methods and allows generalizing Mirror Descent to general
Riemannian geometries, even when the metric tensor is {\em not} a Hessian, and
thus there is no "dual."
