We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: Interpreting learning in biological neural networks as zero-order optimization method

Abstract: Recently, significant progress has been made regarding the statistical understanding of artificial neural networks (ANNs). ANNs are motivated by the functioning of the brain, but differ in several crucial aspects. In particular, the locality in the updating rule of the connection parameters in biological neural networks (BNNs) makes it biologically implausible that the learning of the brain is based on gradient descent. In this work, we look at the brain as a statistical method for supervised learning. The main contribution is to relate the local updating rule of the connection parameters in BNNs to a zero-order optimization method. It is shown that the expected values of the iterates implement a modification of gradient descent.
Subjects: Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE); Statistics Theory (math.ST)
Cite as: arXiv:2301.11777 [cs.LG]
  (or arXiv:2301.11777v2 [cs.LG] for this version)

Submission history

From: Johannes Schmidt-Hieber [view email]
[v1] Fri, 27 Jan 2023 15:30:25 GMT (248kb,D)
[v2] Thu, 23 Mar 2023 13:28:58 GMT (252kb,D)

Link back to: arXiv, form interface, contact.