References & Citations
Computer Science > Artificial Intelligence
Title: Prioritized Sweeping Neural DynaQ with Multiple Predecessors, and Hippocampal Replays
(Submitted on 15 Feb 2018 (v1), last revised 13 Aug 2018 (this version, v2))
Abstract: During sleep and awake rest, the hippocampus replays sequences of place cells that have been activated during prior experiences. These have been interpreted as a memory consolidation process, but recent results suggest a possible interpretation in terms of reinforcement learning. The Dyna reinforcement learning algorithms use off-line replays to improve learning. Under limited replay budget, a prioritized sweeping approach, which requires a model of the transitions to the predecessors, can be used to improve performance. We investigate whether such algorithms can explain the experimentally observed replays. We propose a neural network version of prioritized sweeping Q-learning, for which we developed a growing multiple expert algorithm, able to cope with multiple predecessors. The resulting architecture is able to improve the learning of simulated agents confronted to a navigation task. We predict that, in animals, learning the world model should occur during rest periods, and that the corresponding replays should be shuffled.
Submission history
From: Benoît Girard [view email][v1] Thu, 15 Feb 2018 15:15:19 GMT (418kb,D)
[v2] Mon, 13 Aug 2018 12:27:55 GMT (636kb,D)
Link back to: arXiv, form interface, contact.