We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

q-bio.NC

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Quantitative Biology > Neurons and Cognition

Title: Relaxing the Constraints on Predictive Coding Models

Abstract: Predictive coding is an influential theory of cortical function which posits that the principal computation the brain performs, which underlies both perception and learning, is the minimization of prediction errors. While motivated by high-level notions of variational inference, detailed neurophysiological models of cortical microcircuits which can implements its computations have been developed. Moreover, under certain conditions, predictive coding has been shown to approximate the backpropagation of error algorithm, and thus provides a relatively biologically plausible credit-assignment mechanism for training deep networks. However, standard implementations of the algorithm still involve potentially neurally implausible features such as identical forward and backward weights, backward nonlinear derivatives, and 1-1 error unit connectivity. In this paper, we show that these features are not integral to the algorithm and can be removed either directly or through learning additional sets of parameters with Hebbian update rules without noticeable harm to learning performance. Our work thus relaxes current constraints on potential microcircuit designs and hopefully opens up new regions of the design-space for neuromorphic implementations of predictive coding.
Comments: 02/10/20 initial upload; 10/10/20 minor fixes
Subjects: Neurons and Cognition (q-bio.NC); Artificial Intelligence (cs.AI); Machine Learning (stat.ML)
Cite as: arXiv:2010.01047 [q-bio.NC]
  (or arXiv:2010.01047v2 [q-bio.NC] for this version)

Submission history

From: Beren Millidge Mr [view email]
[v1] Fri, 2 Oct 2020 15:21:37 GMT (1677kb,D)
[v2] Sat, 10 Oct 2020 14:09:12 GMT (1677kb,D)

Link back to: arXiv, form interface, contact.