We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

math.AP

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Analysis of PDEs

Title: Stability for the Training of Deep Neural Networks and Other Classifiers

Abstract: We examine the stability of loss-minimizing training processes that are used for deep neural network (DNN) and other classifiers. While a classifier is optimized during training through a so-called loss function, the performance of classifiers is usually evaluated by some measure of accuracy, such as the overall accuracy which quantifies the proportion of objects that are well classified. This leads to the guiding question of stability: does decreasing loss through training always result in increased accuracy? We formalize the notion of stability, and provide examples of instability. Our main result is two novel conditions on the classifier which, if either is satisfied, ensure stability of training, that is we derive tight bounds on accuracy as loss decreases. These conditions are explicitly verifiable in practice on a given dataset. Our results do not depend on the algorithm used for training, as long as loss decreases with training.
Comments: 25 pages; 1 Figure
Subjects: Analysis of PDEs (math.AP); Machine Learning (cs.LG)
Cite as: arXiv:2002.04122 [math.AP]
  (or arXiv:2002.04122v1 [math.AP] for this version)

Submission history

From: Clarke Safsten [view email]
[v1] Mon, 10 Feb 2020 22:48:13 GMT (56kb,D)

Link back to: arXiv, form interface, contact.