Current browse context:
stat.ML
Change to browse by:
References & Citations
Statistics > Machine Learning
Title: On Quadratic Penalties in Elastic Weight Consolidation
(Submitted on 11 Dec 2017)
Abstract: Elastic weight consolidation (EWC, Kirkpatrick et al, 2017) is a novel algorithm designed to safeguard against catastrophic forgetting in neural networks. EWC can be seen as an approximation to Laplace propagation (Eskin et al, 2004), and this view is consistent with the motivation given by Kirkpatrick et al (2017). In this note, I present an extended derivation that covers the case when there are more than two tasks. I show that the quadratic penalties in EWC are inconsistent with this derivation and might lead to double-counting data from earlier tasks.
Link back to: arXiv, form interface, contact.