We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.LG

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: On the uncertainty principle of neural networks

Abstract: Despite the successes in many fields, it is found that neural networks are difficult to be both accurate and robust, i.e., high accuracy networks are often vulnerable. Various empirical and analytic studies have substantiated that there is more or less a trade-off between the accuracy and robustness of neural networks. If the property is inherent, applications based on the neural networks are vulnerable with untrustworthy predictions. To more deeply explore and understand this issue, in this study we show that the accuracy-robustness trade-off is an intrinsic property whose underlying mechanism is closely related to the uncertainty principle in quantum mechanics. By relating the loss function in neural networks to the wave function in quantum mechanics, we show that the inputs and their conjugates cannot be resolved by a neural network simultaneously. This work thus provides an insightful explanation for the inevitability of the accuracy-robustness dilemma for general deep networks from an entirely new perspective, and furthermore, reveals a potential possibility to study various properties of neural networks with the mature mathematical tools in quantum physics.
Comments: 8 pages, 5 figures
Subjects: Machine Learning (cs.LG); Computational Physics (physics.comp-ph); Quantum Physics (quant-ph)
Cite as: arXiv:2205.01493 [cs.LG]
  (or arXiv:2205.01493v3 [cs.LG] for this version)

Submission history

From: Jun-Jie Zhang [view email]
[v1] Tue, 3 May 2022 13:48:12 GMT (1377kb,D)
[v2] Wed, 26 Oct 2022 12:18:42 GMT (421kb,D)
[v3] Thu, 27 Oct 2022 04:23:45 GMT (2072kb,D)

Link back to: arXiv, form interface, contact.