We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.LG

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: A mathematical model for automatic differentiation in machine learning

Authors: Jerome Bolte (TSE), Edouard Pauwels (IRIT-ADRIA)
Abstract: Automatic differentiation, as implemented today, does not have a simple mathematical model adapted to the needs of modern machine learning. In this work we articulate the relationships between differentiation of programs as implemented in practice and differentiation of nonsmooth functions. To this end we provide a simple class of functions, a nonsmooth calculus, and show how they apply to stochastic approximation methods. We also evidence the issue of artificial critical points created by algorithmic differentiation and show how usual methods avoid these points with probability one.
Subjects: Machine Learning (cs.LG); Optimization and Control (math.OC); Machine Learning (stat.ML)
Journal reference: Conference on Neural Information Processing Systems, Dec 2020, Vancouver, Canada
Cite as: arXiv:2006.02080 [cs.LG]
  (or arXiv:2006.02080v2 [cs.LG] for this version)

Submission history

From: Edouard Pauwels [view email]
[v1] Wed, 3 Jun 2020 07:33:29 GMT (106kb,D)
[v2] Thu, 29 Oct 2020 15:11:30 GMT (136kb,D)

Link back to: arXiv, form interface, contact.