We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ML

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Machine Learning

Title: Hierarchical Indian Buffet Neural Networks for Bayesian Continual Learning

Abstract: We place an Indian Buffet process (IBP) prior over the structure of a Bayesian Neural Network (BNN), thus allowing the complexity of the BNN to increase and decrease automatically. We further extend this model such that the prior on the structure of each hidden layer is shared globally across all layers, using a Hierarchical-IBP (H-IBP). We apply this model to the problem of resource allocation in Continual Learning (CL) where new tasks occur and the network requires extra resources. Our model uses online variational inference with reparameterisation of the Bernoulli and Beta distributions which constitute the IBP and H-IBP priors. As we automatically learn the number of weights in each layer of the BNN, overfitting and underfitting problems are largely overcome. We show empirically that our approach offers a competitive edge over existing methods in CL.
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)
Cite as: arXiv:1912.02290 [stat.ML]
  (or arXiv:1912.02290v3 [stat.ML] for this version)

Submission history

From: Samuel Kessler [view email]
[v1] Wed, 4 Dec 2019 22:43:31 GMT (313kb,D)
[v2] Thu, 20 Feb 2020 15:48:35 GMT (448kb,D)
[v3] Fri, 28 Feb 2020 19:54:01 GMT (183kb,D)

Link back to: arXiv, form interface, contact.