We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:


Current browse context:


Change to browse by:

References & Citations


(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Quantitative Biology > Neurons and Cognition

Title: Tensor decomposition of higher-order correlations by nonlinear Hebbian plasticity

Abstract: Biological synaptic plasticity exhibits nonlinearities that are not accounted for by classic Hebbian learning rules. Here, we introduce a simple family of generalized nonlinear Hebbian learning rules. We study the computations implemented by their dynamics in the simple setting of a neuron receiving feedforward inputs. These nonlinear Hebbian rules allow a neuron to learn tensor decompositions of its higher-order input correlations. The particular input correlation decomposed and the form of the decomposition depend on the location of nonlinearities in the plasticity rule. For simple, biologically motivated parameters, the neuron learns eigenvectors of higher-order input correlation tensors. We prove that tensor eigenvectors are attractors and determine their basins of attraction. We calculate the volume of those basins, showing that the dominant eigenvector has the largest basin of attraction. We then study arbitrary learning rules and find that any learning rule that admits a finite Taylor expansion into the neural input and output also has stable equilibria at generalized eigenvectors of higher-order input correlation tensors. Nonlinearities in synaptic plasticity thus allow a neuron to encode higher-order input correlations in a simple fashion.
Comments: 9 pages, 3 figures + supplemental 20 pages, 4 figures; Neurips 2021
Subjects: Neurons and Cognition (q-bio.NC); Machine Learning (stat.ML)
Cite as: arXiv:2106.15685 [q-bio.NC]
  (or arXiv:2106.15685v3 [q-bio.NC] for this version)

Submission history

From: Gabriel Ocker [view email]
[v1] Tue, 29 Jun 2021 19:24:35 GMT (1700kb,D)
[v2] Tue, 17 Aug 2021 16:48:17 GMT (1704kb,D)
[v3] Mon, 25 Oct 2021 20:39:43 GMT (1702kb,D)

Link back to: arXiv, form interface, contact.