We gratefully acknowledge support from
the Simons Foundation and member institutions.

Neurons and Cognition

New submissions

[ total of 8 entries: 1-8 ]
[ showing up to 2000 entries per page: fewer | more ]

New submissions for Fri, 27 Mar 20

[1]  arXiv:2003.11660 [pdf, other]
Title: R-FORCE: Robust Learning for Random Recurrent Neural Networks
Comments: Github Repository: this https URL
Subjects: Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE); Neurons and Cognition (q-bio.NC)

Random Recurrent Neural Networks (RRNN) are the simplest recurrent networks to model and extract features from sequential data. The simplicity however comes with a price; RRNN are known to be susceptible to diminishing/exploding gradient problem when trained with gradient-descent based optimization. To enhance robustness of RRNN, alternative training approaches have been proposed. Specifically, FORCE learning approach proposed a recursive least squares alternative to train RRNN and was shown to be applicable even for the challenging task of target-learning, where the network is tasked with generating dynamic patterns with no guiding input. While FORCE training indicates that solving target-learning is possible, it appears to be effective only in a specific regime of network dynamics (edge-of-chaos). We thereby investigate whether initialization of RRNN connectivity according to a tailored distribution can guarantee robust FORCE learning. We are able to generate such distribution by inference of four generating principles constraining the spectrum of the network Jacobian to remain in stability region. This initialization along with FORCE learning provides a robust training method, i.e., Robust-FORCE (R-FORCE). We validate R-FORCE performance on various target functions for a wide range of network configurations and compare with alternative methods. Our experiments indicate that R-FORCE facilitates significantly more stable and accurate target-learning for a wide class of RRNN. Such stability becomes critical in modeling multi-dimensional sequences as we demonstrate on modeling time-series of human body joints during physical movements.

[2]  arXiv:2003.11668 [pdf, other]
Title: Cognitive computation using neural representations of time and space in the Laplace domain
Subjects: Neurons and Cognition (q-bio.NC)

Memory for the past makes use of a record of what happened when---a function over past time. Time cells in the hippocampus and temporal context cells in the entorhinal cortex both code for events as a function of past time, but with very different receptive fields. Time cells in the hippocampus can be understood as a compressed estimate of events as a function of the past. Temporal context cells in the entorhinal cortex can be understood as the Laplace transform of that function, respectively. Other functional cell types in the hippocampus and related regions, including border cells, place cells, trajectory coding, splitter cells, can be understood as coding for functions over space or past movements or their Laplace transforms. More abstract quantities, like distance in an abstract conceptual space or numerosity could also be mapped onto populations of neurons coding for the Laplace transform of functions over those variables. Quantitative cognitive models of memory and evidence accumulation can also be specified in this framework allowing constraints from both behavior and neurophysiology. More generally, the computational power of the Laplace domain could be important for efficiently implementing data-independent operators, which could serve as a basis for neural models of a very broad range of cognitive computations.

[3]  arXiv:2003.11859 [pdf, other]
Title: Thalamo-cortical spiking model of incremental learning combining perception, context and NREM-sleep-mediated noise-resilience
Subjects: Neurons and Cognition (q-bio.NC); Distributed, Parallel, and Cluster Computing (cs.DC)

The brain exhibits capabilities of fast incremental learning from few noisy examples, as well as the ability to associate similar memories in autonomously-created categories and to combine contextual hints with sensory perceptions. Together with sleep, these mechanisms are thought to be key components of many high-level cognitive functions. Yet, little is known about the underlying processes and the specific roles of different brain states. In this work, we exploited the combination of context and perception in a thalamo-cortical model based on a soft winner-take-all circuit of excitatory and inhibitory spiking neurons. After calibrating this model to express awake and deep-sleep states with features comparable with biological measures, we demonstrate the model capability of fast incremental learning from few examples, its resilience when proposed with noisy perceptions and contextual signals, and an improvement in visual classification after sleep due to induced synaptic homeostasis and association of similar memories.

Cross-lists for Fri, 27 Mar 20

[4]  arXiv:2003.11640 (cross-list from cs.NE) [pdf, other]
Title: Transfer between long-term and short-term memory using Conceptors
Authors: Anthony Strock (Mnemosyne, LaBRI, IMN), Nicolas Rougier (Mnemosyne, LaBRI, IMN), Xavier Hinaut (Mnemosyne, LaBRI, IMN)
Subjects: Neural and Evolutionary Computing (cs.NE); Machine Learning (cs.LG); Adaptation and Self-Organizing Systems (nlin.AO); Neurons and Cognition (q-bio.NC); Machine Learning (stat.ML)

We introduce a recurrent neural network model of working memory combining short-term and long-term components. e short-term component is modelled using a gated reservoir model that is trained to hold a value from an input stream when a gate signal is on. e long-term component is modelled using conceptors in order to store inner temporal patterns (that corresponds to values). We combine these two components to obtain a model where information can go from long-term memory to short-term memory and vice-versa and we show how standard operations on conceptors allow to combine long-term memories and describe their effect on short-term memory.

[5]  arXiv:2003.11797 (cross-list from cs.CV) [pdf]
Title: Neural encoding and interpretation for high-level visual cortices based on fMRI using image caption features
Subjects: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Image and Video Processing (eess.IV); Neurons and Cognition (q-bio.NC)

On basis of functional magnetic resonance imaging (fMRI), researchers are devoted to designing visual encoding models to predict the neuron activity of human in response to presented image stimuli and analyze inner mechanism of human visual cortices. Deep network structure composed of hierarchical processing layers forms deep network models by learning features of data on specific task through big dataset. Deep network models have powerful and hierarchical representation of data, and have brought about breakthroughs for visual encoding, while revealing hierarchical structural similarity with the manner of information processing in human visual cortices. However, previous studies almost used image features of those deep network models pre-trained on classification task to construct visual encoding models. Except for deep network structure, the task or corresponding big dataset is also important for deep network models, but neglected by previous studies. Because image classification is a relatively fundamental task, it is difficult to guide deep network models to master high-level semantic representations of data, which causes into that encoding performance for high-level visual cortices is limited. In this study, we introduced one higher-level vision task: image caption (IC) task and proposed the visual encoding model based on IC features (ICFVEM) to encode voxels of high-level visual cortices. Experiment demonstrated that ICFVEM obtained better encoding performance than previous deep network models pre-trained on classification task. In addition, the interpretation of voxels was realized to explore the detailed characteristics of voxels based on the visualization of semantic words, and comparative analysis implied that high-level visual cortices behaved the correlative representation of image content.

[6]  arXiv:2003.11996 (cross-list from cs.NE) [pdf, other]
Title: Accelerated Analog Neuromorphic Computing
Subjects: Neural and Evolutionary Computing (cs.NE); Disordered Systems and Neural Networks (cond-mat.dis-nn); Neurons and Cognition (q-bio.NC)

This paper presents the concepts behind the BrainScales (BSS) accelerated analog neuromorphic computing architecture. It describes the second-generation BrainScales-2 (BSS-2) version and its most recent in-silico realization, the HICANN-X Application Specific Integrated Circuit (ASIC), as it has been developed as part of the neuromorphic computing activities within the European Human Brain Project (HBP). While the first generation is implemented in an 180nm process, the second generation uses 65nm technology. This allows the integration of a digital plasticity processing unit, a highly-parallel micro processor specially built for the computational needs of learning in an accelerated analog neuromorphic systems. The presented architecture is based upon a continuous-time, analog, physical model implementation of neurons and synapses, resembling an analog neuromorphic accelerator attached to build-in digital compute cores. While the analog part emulates the spike-based dynamics of the neural network in continuous-time, the latter simulates biological processes happening on a slower time-scale, like structural and parameter changes. Compared to biological time-scales, the emulation is highly accelerated, i.e. all time-constants are several orders of magnitude smaller than in biology. Programmable ion channel emulation and inter-compartmental conductances allow the modeling of nonlinear dendrites, back-propagating action-potentials as well as NMDA and Calcium plateau potentials. To extend the usability of the analog accelerator, it also supports vector-matrix multiplication. Thereby, BSS-2 supports inference of deep convolutional networks as well as local-learning with complex ensembles of spiking neurons within the same substrate.

Replacements for Fri, 27 Mar 20

[7]  arXiv:2003.11325 (replaced) [pdf, other]
Title: Effect of Diverse Recoding of Granule Cells on Optokinetic Response in A Cerebellar Ring Network with Synaptic Plasticity
Subjects: Neurons and Cognition (q-bio.NC); Biological Physics (physics.bio-ph)
[8]  arXiv:1906.00926 (replaced) [pdf, other]
Title: Human information processing in complex networks
Comments: 87 pages, 26 figures, 13 tables
Subjects: Physics and Society (physics.soc-ph); Biological Physics (physics.bio-ph); Neurons and Cognition (q-bio.NC)
[ total of 8 entries: 1-8 ]
[ showing up to 2000 entries per page: fewer | more ]

Disable MathJax (What is MathJax?)

Links to: arXiv, form interface, find, q-bio, recent, 2003, contact, help  (Access key information)