Geometric Optimisation on Manifolds with Applications to Deep Learning
  We design and implement a Python library to help the non-expert using all
these powerful tools in a way that is efficient, extensible, and simple to
incorporate into the workflow of the data scientist, practitioner, and applied
researcher. The algorithms implemented in this library have been designed with
usability and GPU efficiency in mind, and they can be added to any PyTorch
model with just one extra line of code.
  We showcase the effectiveness of these tools on an application of
optimisation on manifolds in the setting of time series analysis. In this
setting, orthogonal and unitary optimisation is used to constraint and
regularise recurrent models and avoid vanishing and exploding gradient
problems. The algorithms designed for GeoTorch allow us to achieve state of the
art results in the standard tests for this family of models.
  We use tools from comparison geometry to give bounds on quantities that are
of interest in optimisation problems. In particular, we build on the work of
(Kaul 1976) to give explicit bounds on the norm of the second derivative of the
Riemannian exponential.
