We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.NE

Change to browse by:

cs

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Neural and Evolutionary Computing

Title: Reservoir Stack Machines

Abstract: Memory-augmented neural networks equip a recurrent neural network with an explicit memory to support tasks that require information storage without interference over long times. A key motivation for such research is to perform classic computation tasks, such as parsing. However, memory-augmented neural networks are notoriously hard to train, requiring many backpropagation epochs and a lot of data. In this paper, we introduce the reservoir stack machine, a model which can provably recognize all deterministic context-free languages and circumvents the training problem by training only the output layer of a recurrent net and employing auxiliary information during training about the desired interaction with a stack. In our experiments, we validate the reservoir stack machine against deep and shallow networks from the literature on three benchmark tasks for Neural Turing machines and six deterministic context-free languages. Our results show that the reservoir stack machine achieves zero error, even on test sequences longer than the training data, requiring only a few seconds of training time and 100 training sequences.
Comments: in print at the Journal Neurocomputing
Subjects: Neural and Evolutionary Computing (cs.NE)
DOI: 10.1016/j.neucom.2021.05.106
Cite as: arXiv:2105.01616 [cs.NE]
  (or arXiv:2105.01616v2 [cs.NE] for this version)

Submission history

From: Benjamin Paassen [view email]
[v1] Tue, 4 May 2021 16:50:40 GMT (41kb)
[v2] Mon, 26 Jul 2021 08:08:30 GMT (41kb)

Link back to: arXiv, form interface, contact.