We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.LG

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo

Computer Science > Machine Learning

Title: Efficient anti-symmetrization of a neural network layer by taming the sign problem

Abstract: Explicit antisymmetrization of a neural network is a potential candidate for a universal function approximator for generic antisymmetric functions, which are ubiquitous in quantum physics. However, this procedure is a priori factorially costly to implement, making it impractical for large numbers of particles. The strategy also suffers from a sign problem. Namely, due to near-exact cancellation of positive and negative contributions, the magnitude of the antisymmetrized function may be significantly smaller than before anti-symmetrization. We show that the anti-symmetric projection of a two-layer neural network can be evaluated efficiently, opening the door to using a generic antisymmetric layer as a building block in anti-symmetric neural network Ansatzes. This approximation is effective when the sign problem is controlled, and we show that this property depends crucially the choice of activation function under standard Xavier/He initialization methods. As a consequence, using a smooth activation function requires re-scaling of the neural network weights compared to standard initializations.
Comments: To appear in JML, ISSN: 2790-2048(e), 2790-203X(p)
Subjects: Machine Learning (cs.LG); Numerical Analysis (math.NA); Quantum Physics (quant-ph)
Cite as: arXiv:2205.12250 [cs.LG]
  (or arXiv:2205.12250v2 [cs.LG] for this version)

Submission history

From: Nilin Abrahamsen [view email]
[v1] Tue, 24 May 2022 17:56:56 GMT (97kb,D)
[v2] Thu, 7 Sep 2023 00:44:17 GMT (113kb,D)

Link back to: arXiv, form interface, contact.