We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cond-mat.soft

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Condensed Matter > Soft Condensed Matter

Title: Learning Without a Global Clock: Asynchronous Learning in a Physics-Driven Learning Network

Abstract: In a neuron network, synapses update individually using local information, allowing for entirely decentralized learning. In contrast, elements in an artificial neural network (ANN) are typically updated simultaneously using a central processor. Here we investigate the feasibility and effect of asynchronous learning in a recently introduced decentralized, physics-driven learning network. We show that desynchronizing the learning process does not degrade performance for a variety of tasks in an idealized simulation. In experiment, desynchronization actually improves performance by allowing the system to better explore the discretized state space of solutions. We draw an analogy between asynchronicity and mini-batching in stochastic gradient descent, and show that they have similar effects on the learning process. Desynchronizing the learning process establishes physics-driven learning networks as truly fully distributed learning machines, promoting better performance and scalability in deployment.
Comments: 5 pages 4 figures
Subjects: Soft Condensed Matter (cond-mat.soft); Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE)
DOI: 10.1063/5.0084631
Cite as: arXiv:2201.04626 [cond-mat.soft]
  (or arXiv:2201.04626v1 [cond-mat.soft] for this version)

Submission history

From: Sam Dillavou [view email]
[v1] Mon, 10 Jan 2022 05:38:01 GMT (4663kb,D)

Link back to: arXiv, form interface, contact.