Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Pay attention to your loss: understanding misconceptions about 1-Lipschitz neural networks
(Submitted on 11 Apr 2021 (v1), last revised 17 Oct 2022 (this version, v6))
Abstract: Lipschitz constrained networks have gathered considerable attention in the deep learning community, with usages ranging from Wasserstein distance estimation to the training of certifiably robust classifiers. However they remain commonly considered as less accurate, and their properties in learning are still not fully understood. In this paper we clarify the matter: when it comes to classification 1-Lipschitz neural networks enjoy several advantages over their unconstrained counterpart. First, we show that these networks are as accurate as classical ones, and can fit arbitrarily difficult boundaries. Then, relying on a robustness metric that reflects operational needs we characterize the most robust classifier: the WGAN discriminator. Next, we show that 1-Lipschitz neural networks generalize well under milder assumptions. Finally, we show that hyper-parameters of the loss are crucial for controlling the accuracy-robustness trade-off. We conclude that they exhibit appealing properties to pave the way toward provably accurate, and provably robust neural networks.
Submission history
From: Louis Béthune [view email][v1] Sun, 11 Apr 2021 20:31:32 GMT (5167kb,D)
[v2] Tue, 13 Apr 2021 10:15:02 GMT (5167kb,D)
[v3] Tue, 18 May 2021 20:27:55 GMT (5167kb,D)
[v4] Thu, 27 May 2021 10:01:49 GMT (2725kb,D)
[v5] Fri, 3 Jun 2022 15:05:34 GMT (9593kb,D)
[v6] Mon, 17 Oct 2022 10:03:41 GMT (18138kb,D)
Link back to: arXiv, form interface, contact.