We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cond-mat.dis-nn

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Condensed Matter > Disordered Systems and Neural Networks

Title: Transient chaotic dimensionality expansion by recurrent networks

Abstract: Cortical neurons communicate with spikes, which are discrete events in time. Even if the timings of the individual events are strongly chaotic (microscopic chaos), the rate of events might still be non-chaotic or at the edge of what is known as rate chaos. Such edge-of-chaos dynamics are beneficial to the computational power of neuronal networks. We analyze both types of chaotic dynamics in densely connected networks of asynchronous binary neurons, by developing and applying a model-independent field theory for neuronal networks. We find a strongly size-dependent transition to microscopic chaos. We then expose the conceptual difficulty at the heart of the definition of rate chaos, identify two reasonable definitions, and show that for neither of them the binary network dynamics crosses a transition to rate chaos.
The analysis of diverging trajectories in chaotic networks also allows us to study classification of linearly non-separable classes of stimuli in a reservoir computing approach. We show that microscopic chaos rapidly expands the dimensionality of the representation while, crucially, the number of dimensions corrupted by noise lags behind. This translates to a transient peak in the networks' classification performance even deeply in the chaotic regime, challenging the view that computational performance is always optimal near the edge of chaos. This is a general effect in high dimensional chaotic systems, and not specific to binary networks: We also demonstrate it in a continuous 'rate' network, a spiking LIF network, and an LSTM network. For binary and LIF networks, classification performance peaks rapidly within one activation per participating neuron, demonstrating fast event-based computation that may be exploited by biological neural systems, for which we propose testable predictions.
Comments: Fully overhauled version with lots of new material
Subjects: Disordered Systems and Neural Networks (cond-mat.dis-nn); Neurons and Cognition (q-bio.NC)
Cite as: arXiv:2002.11006 [cond-mat.dis-nn]
  (or arXiv:2002.11006v3 [cond-mat.dis-nn] for this version)

Submission history

From: Christian Keup [view email]
[v1] Tue, 25 Feb 2020 16:26:29 GMT (1567kb)
[v2] Wed, 6 May 2020 15:04:30 GMT (1306kb)
[v3] Tue, 24 Nov 2020 12:44:23 GMT (1927kb)

Link back to: arXiv, form interface, contact.