Current browse context:
stat.ML
Change to browse by:
References & Citations
Statistics > Machine Learning
Title: Variational Auto-Regressive Gaussian Processes for Continual Learning
(Submitted on 9 Jun 2020 (v1), last revised 12 Jun 2021 (this version, v3))
Abstract: Through sequential construction of posteriors on observing data online, Bayes' theorem provides a natural framework for continual learning. We develop Variational Auto-Regressive Gaussian Processes (VAR-GPs), a principled posterior updating mechanism to solve sequential tasks in continual learning. By relying on sparse inducing point approximations for scalable posteriors, we propose a novel auto-regressive variational distribution which reveals two fruitful connections to existing results in Bayesian inference, expectation propagation and orthogonal inducing points. Mean predictive entropy estimates show VAR-GPs prevent catastrophic forgetting, which is empirically supported by strong performance on modern continual learning benchmarks against competitive baselines. A thorough ablation study demonstrates the efficacy of our modeling choices.
Submission history
From: Sanyam Kapoor [view email][v1] Tue, 9 Jun 2020 19:23:57 GMT (3332kb,D)
[v2] Tue, 25 May 2021 13:02:34 GMT (811kb,D)
[v3] Sat, 12 Jun 2021 06:13:39 GMT (810kb,D)
Link back to: arXiv, form interface, contact.