Disordered Systems and Neural Networks
New submissions
[ showing up to 2000 entries per page: fewer  more ]
New submissions for Thu, 26 Nov 20
 [1] arXiv:2011.12428 [pdf, other]

Title: The dynamics of learning with feedback alignmentComments: The accompanying code for this paper is available at this https URLSubjects: Machine Learning (stat.ML); Disordered Systems and Neural Networks (condmat.disnn); Statistical Mechanics (condmat.statmech); Machine Learning (cs.LG)
Direct Feedback Alignment (DFA) is emerging as an efficient and biologically plausible alternative to the ubiquitous backpropagation algorithm for training deep neural networks. Despite relying on random feedback weights for the backward pass, DFA successfully trains stateoftheart models such as Transformers. On the other hand, it notoriously fails to train convolutional networks. An understanding of the inner workings of DFA to explain these diverging results remains elusive. Here, we propose a theory for the success of DFA. We first show that learning in shallow networks proceeds in two steps: an alignment phase, where the model adapts its weights to align the approximate gradient with the true gradient of the loss function, is followed by a memorisation phase, where the model focuses on fitting the data. This twostep process has a degeneracy breaking effect: out of all the lowloss solutions in the landscape, a network trained with DFA naturally converges to the solution which maximises gradient alignment. We also identify a key quantity underlying alignment in deep linear networks: the conditioning of the alignment matrices. The latter enables a detailed understanding of the impact of data structure on alignment, and suggests a simple explanation for the wellknown failure of DFA to train convolutional neural networks. Numerical experiments on MNIST and CIFAR10 clearly demonstrate degeneracy breaking in deep nonlinear networks and show that the alignthenmemorize process occurs sequentially from the bottom layers of the network to the top.
 [2] arXiv:2011.12453 [pdf, other]

Title: Scalable variational Monte Carlo with graph neural ansatzComments: 9 pages, 3 figures, NeurIPS 2020 workshop on Machine Learning and the Physical SciencesSubjects: Strongly Correlated Electrons (condmat.strel); Disordered Systems and Neural Networks (condmat.disnn)
Deep neural networks have been shown as a potentially powerful ansatz in variational Monte Carlo for solving quantum manybody problems. We propose two improvements in this direction. The first is graph neural ansatz (GNA), which is a variational wavefunction universal to arbitrary geometry. GNA results in accurate groundstate energies on 2D Kagome lattices, triangular lattices, and randomly connected graphs. Secondly, we design a distributed workflow on multiple accelerators to scale up the computation. We compute Kagome lattices with sizes up to 432 sites on 128 TPU cores. The parameter sharing nature of the GNA also leads to transferability across different system sizes and geometries.
 [3] arXiv:2011.12489 [pdf, other]

Title: Gasliquid phase separation at zero temperature: mechanical interpretation and implications for gelationComments: 13 pages, 11 figuresSubjects: Soft Condensed Matter (condmat.soft); Disordered Systems and Neural Networks (condmat.disnn)
The relation between glasses and gels has been intensely debated for decades; however, the transition between these two phases remains elusive. In this study, we conducted numerical experiments of athermal quasistatic decompression to investigate a gasliquid phase separation, which is one of the main gel formation processes. A normal mode analysis revealed that the phase separation is signaled by the vanishing of the lowest eigenenergy, similar to plastic events under shear. One primary difference from the shearinduced plasticity is that the vanishing mode experiences a qualitative change in its spatial energy distribution at the phase separation point. These findings enable us to define gelation based on mechanics.
Replacements for Thu, 26 Nov 20
 [4] arXiv:2003.04741 (replaced) [pdf, other]

Title: Persistence of hierarchical network organization and emergent topologies in models of functional connectivityComments: 5 figuresSubjects: Disordered Systems and Neural Networks (condmat.disnn); Physics and Society (physics.socph); Neurons and Cognition (qbio.NC)
 [5] arXiv:2010.15840 (replaced) [pdf, other]

Title: Postselectionfree entanglement dynamics via spacetime dualityComments: 4 pages main text + 9 page supplemental material. v2: added references, expanded supplement, minor modifications to textSubjects: Disordered Systems and Neural Networks (condmat.disnn); Statistical Mechanics (condmat.statmech); Quantum Physics (quantph)
 [6] arXiv:2011.10899 (replaced) [pdf, other]

Title: Finite size effects in the microscopic critical properties of jammed configurations: a comprehensive study of the effects of different types of disorderAuthors: Patrick Charbonneau, Eric Corwin, Cameron Dennis, Rafael Díaz Hernández Rojas, Harukuni Ikeda, Giorgio Parisi, Federico RicciTersenghiComments: 16 pages, 7 figures; corrected typosSubjects: Disordered Systems and Neural Networks (condmat.disnn); Soft Condensed Matter (condmat.soft); Statistical Mechanics (condmat.statmech)
 [7] arXiv:2011.11336 (replaced) [pdf, other]

Title: Harnessing adaptive dynamics in neuromemristive nanowire networks for transfer learningAuthors: Ruomin Zhu, Joel Hochstetter, Alon Loeffler, Adrian DiazAlvarez, Adam Stieg, James Gimzewski, Tomonobu Nakayama, Zdenka KuncicSubjects: Disordered Systems and Neural Networks (condmat.disnn)
 [8] arXiv:1807.03995 (replaced) [pdf, other]

Title: Effective Number Theory: Counting the Identities of a Quantum StateComments: 19 pages, 1 figure; v2: some restructuring and presentation improvements, 15 pages, 1 figure; v3: published versionJournalref: Entropy 22(11), 2020, 1273Subjects: Quantum Physics (quantph); Disordered Systems and Neural Networks (condmat.disnn); High Energy Physics  Lattice (heplat); High Energy Physics  Theory (hepth); Mathematical Physics (mathph)
 [9] arXiv:1905.03573 (replaced) [pdf, other]

Title: Phase domain boundary motion and memristance in gradientdoped FeRh nanopillars induced by spin injectionAuthors: Rowan C. Temple, Mark C. Rosamond, Jamie R. Massey, Trevor P. Almeida, Edmund H. Linfield, Damien McGrouther, Stephen McVitie, Thomas A. Moore, Christopher H. MarrowsSubjects: Materials Science (condmat.mtrlsci); Disordered Systems and Neural Networks (condmat.disnn); Applied Physics (physics.appph)
 [10] arXiv:2005.12890 (replaced) [pdf, other]

Title: A reinforcement learning approach to rare trajectory samplingComments: 55+6 pages, 7+1 figuresSubjects: Statistical Mechanics (condmat.statmech); Disordered Systems and Neural Networks (condmat.disnn); Machine Learning (cs.LG)
[ showing up to 2000 entries per page: fewer  more ]
Disable MathJax (What is MathJax?)
Links to: arXiv, form interface, find, condmat, recent, 2011, contact, help (Access key information)