We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ML

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Machine Learning

Title: Continual Learning with Distributed Optimization: Does CoCoA Forget?

Abstract: We focus on the continual learning problem where the tasks arrive sequentially and the aim is to perform well on the newly arrived task without performance degradation on the previously seen tasks. In contrast to the continual learning literature focusing on the centralized setting, we investigate the distributed estimation framework. We consider the well-established distributed learning algorithm CoCoA. We derive closed form expressions for the iterations for the overparametrized case. We illustrate the convergence and the error performance of the algorithm based on the over/under-parametrization of the problem. Our results show that depending on the problem dimensions and data generation assumptions, CoCoA can perform continual learning over a sequence of tasks, i.e., it can learn a new task without forgetting previously learned tasks, with access only to one task at a time.
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Signal Processing (eess.SP)
Cite as: arXiv:2211.16994 [stat.ML]
  (or arXiv:2211.16994v3 [stat.ML] for this version)

Submission history

From: Martin Hellkvist [view email]
[v1] Wed, 30 Nov 2022 13:49:43 GMT (560kb,D)
[v2] Fri, 2 Dec 2022 13:49:14 GMT (560kb,D)
[v3] Thu, 22 Dec 2022 09:08:26 GMT (560kb,D)

Link back to: arXiv, form interface, contact.