We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:


Current browse context:


Change to browse by:

References & Citations


(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Optimization and Control

Title: Geometric Optimisation on Manifolds with Applications to Deep Learning

Abstract: We design and implement a Python library to help the non-expert using all these powerful tools in a way that is efficient, extensible, and simple to incorporate into the workflow of the data scientist, practitioner, and applied researcher. The algorithms implemented in this library have been designed with usability and GPU efficiency in mind, and they can be added to any PyTorch model with just one extra line of code.
We showcase the effectiveness of these tools on an application of optimisation on manifolds in the setting of time series analysis. In this setting, orthogonal and unitary optimisation is used to constraint and regularise recurrent models and avoid vanishing and exploding gradient problems. The algorithms designed for GeoTorch allow us to achieve state of the art results in the standard tests for this family of models.
We use tools from comparison geometry to give bounds on quantities that are of interest in optimisation problems. In particular, we build on the work of (Kaul 1976) to give explicit bounds on the norm of the second derivative of the Riemannian exponential.
Comments: DPhil Thesis. 154 pages. arXiv admin note: text overlap with arXiv:2008.02517
Subjects: Optimization and Control (math.OC); Machine Learning (cs.LG)
Cite as: arXiv:2203.04794 [math.OC]
  (or arXiv:2203.04794v1 [math.OC] for this version)

Submission history

From: Mario Lezcano-Casado [view email]
[v1] Wed, 9 Mar 2022 15:20:07 GMT (2273kb,D)

Link back to: arXiv, form interface, contact.