We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ML

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Machine Learning

Title: Approximating Lipschitz continuous functions with GroupSort neural networks

Abstract: Recent advances in adversarial attacks and Wasserstein GANs have advocated for use of neural networks with restricted Lipschitz constants. Motivated by these observations, we study the recently introduced GroupSort neural networks, with constraints on the weights, and make a theoretical step towards a better understanding of their expressive power. We show in particular how these networks can represent any Lipschitz continuous piecewise linear functions. We also prove that they are well-suited for approximating Lipschitz continuous functions and exhibit upper bounds on both the depth and size. To conclude, the efficiency of GroupSort networks compared with more standard ReLU networks is illustrated in a set of synthetic experiments.
Comments: 16 pages
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)
Cite as: arXiv:2006.05254 [stat.ML]
  (or arXiv:2006.05254v2 [stat.ML] for this version)

Submission history

From: Ugo Tanielian [view email]
[v1] Tue, 9 Jun 2020 13:37:43 GMT (440kb,D)
[v2] Mon, 8 Feb 2021 18:17:50 GMT (1456kb,D)

Link back to: arXiv, form interface, contact.