We gratefully acknowledge support from
the Simons Foundation and member institutions.

Disordered Systems and Neural Networks

New submissions

[ total of 10 entries: 1-10 ]
[ showing up to 2000 entries per page: fewer | more ]

New submissions for Thu, 26 Nov 20

[1]  arXiv:2011.12428 [pdf, other]
Title: The dynamics of learning with feedback alignment
Comments: The accompanying code for this paper is available at this https URL
Subjects: Machine Learning (stat.ML); Disordered Systems and Neural Networks (cond-mat.dis-nn); Statistical Mechanics (cond-mat.stat-mech); Machine Learning (cs.LG)

Direct Feedback Alignment (DFA) is emerging as an efficient and biologically plausible alternative to the ubiquitous backpropagation algorithm for training deep neural networks. Despite relying on random feedback weights for the backward pass, DFA successfully trains state-of-the-art models such as Transformers. On the other hand, it notoriously fails to train convolutional networks. An understanding of the inner workings of DFA to explain these diverging results remains elusive. Here, we propose a theory for the success of DFA. We first show that learning in shallow networks proceeds in two steps: an alignment phase, where the model adapts its weights to align the approximate gradient with the true gradient of the loss function, is followed by a memorisation phase, where the model focuses on fitting the data. This two-step process has a degeneracy breaking effect: out of all the low-loss solutions in the landscape, a network trained with DFA naturally converges to the solution which maximises gradient alignment. We also identify a key quantity underlying alignment in deep linear networks: the conditioning of the alignment matrices. The latter enables a detailed understanding of the impact of data structure on alignment, and suggests a simple explanation for the well-known failure of DFA to train convolutional neural networks. Numerical experiments on MNIST and CIFAR10 clearly demonstrate degeneracy breaking in deep non-linear networks and show that the align-then-memorize process occurs sequentially from the bottom layers of the network to the top.

[2]  arXiv:2011.12453 [pdf, other]
Title: Scalable variational Monte Carlo with graph neural ansatz
Authors: Li Yang, Wenjun Hu, Li Li
Comments: 9 pages, 3 figures, NeurIPS 2020 workshop on Machine Learning and the Physical Sciences
Subjects: Strongly Correlated Electrons (cond-mat.str-el); Disordered Systems and Neural Networks (cond-mat.dis-nn)

Deep neural networks have been shown as a potentially powerful ansatz in variational Monte Carlo for solving quantum many-body problems. We propose two improvements in this direction. The first is graph neural ansatz (GNA), which is a variational wavefunction universal to arbitrary geometry. GNA results in accurate ground-state energies on 2D Kagome lattices, triangular lattices, and randomly connected graphs. Secondly, we design a distributed workflow on multiple accelerators to scale up the computation. We compute Kagome lattices with sizes up to 432 sites on 128 TPU cores. The parameter sharing nature of the GNA also leads to transferability across different system sizes and geometries.

[3]  arXiv:2011.12489 [pdf, other]
Title: Gas-liquid phase separation at zero temperature: mechanical interpretation and implications for gelation
Comments: 13 pages, 11 figures
Subjects: Soft Condensed Matter (cond-mat.soft); Disordered Systems and Neural Networks (cond-mat.dis-nn)

The relation between glasses and gels has been intensely debated for decades; however, the transition between these two phases remains elusive. In this study, we conducted numerical experiments of athermal quasistatic decompression to investigate a gas-liquid phase separation, which is one of the main gel formation processes. A normal mode analysis revealed that the phase separation is signaled by the vanishing of the lowest eigenenergy, similar to plastic events under shear. One primary difference from the shear-induced plasticity is that the vanishing mode experiences a qualitative change in its spatial energy distribution at the phase separation point. These findings enable us to define gelation based on mechanics.

Replacements for Thu, 26 Nov 20

[4]  arXiv:2003.04741 (replaced) [pdf, other]
Title: Persistence of hierarchical network organization and emergent topologies in models of functional connectivity
Comments: 5 figures
Subjects: Disordered Systems and Neural Networks (cond-mat.dis-nn); Physics and Society (physics.soc-ph); Neurons and Cognition (q-bio.NC)
[5]  arXiv:2010.15840 (replaced) [pdf, other]
Title: Postselection-free entanglement dynamics via spacetime duality
Comments: 4 pages main text + 9 page supplemental material. v2: added references, expanded supplement, minor modifications to text
Subjects: Disordered Systems and Neural Networks (cond-mat.dis-nn); Statistical Mechanics (cond-mat.stat-mech); Quantum Physics (quant-ph)
[6]  arXiv:2011.10899 (replaced) [pdf, other]
Title: Finite size effects in the microscopic critical properties of jammed configurations: a comprehensive study of the effects of different types of disorder
Comments: 16 pages, 7 figures; corrected typos
Subjects: Disordered Systems and Neural Networks (cond-mat.dis-nn); Soft Condensed Matter (cond-mat.soft); Statistical Mechanics (cond-mat.stat-mech)
[7]  arXiv:2011.11336 (replaced) [pdf, other]
Title: Harnessing adaptive dynamics in neuro-memristive nanowire networks for transfer learning
Subjects: Disordered Systems and Neural Networks (cond-mat.dis-nn)
[8]  arXiv:1807.03995 (replaced) [pdf, other]
Title: Effective Number Theory: Counting the Identities of a Quantum State
Comments: 19 pages, 1 figure; v2: some restructuring and presentation improvements, 15 pages, 1 figure; v3: published version
Journal-ref: Entropy 22(11), 2020, 1273
Subjects: Quantum Physics (quant-ph); Disordered Systems and Neural Networks (cond-mat.dis-nn); High Energy Physics - Lattice (hep-lat); High Energy Physics - Theory (hep-th); Mathematical Physics (math-ph)
[9]  arXiv:1905.03573 (replaced) [pdf, other]
Title: Phase domain boundary motion and memristance in gradient-doped FeRh nanopillars induced by spin injection
Subjects: Materials Science (cond-mat.mtrl-sci); Disordered Systems and Neural Networks (cond-mat.dis-nn); Applied Physics (physics.app-ph)
[10]  arXiv:2005.12890 (replaced) [pdf, other]
Title: A reinforcement learning approach to rare trajectory sampling
Comments: 55+6 pages, 7+1 figures
Subjects: Statistical Mechanics (cond-mat.stat-mech); Disordered Systems and Neural Networks (cond-mat.dis-nn); Machine Learning (cs.LG)
[ total of 10 entries: 1-10 ]
[ showing up to 2000 entries per page: fewer | more ]

Disable MathJax (What is MathJax?)

Links to: arXiv, form interface, find, cond-mat, recent, 2011, contact, help  (Access key information)