We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ML

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Machine Learning

Title: Generating Neural Networks with Neural Networks

Authors: Lior Deutsch
Abstract: Hypernetworks are neural networks that generate weights for another neural network. We formulate the hypernetwork training objective as a compromise between accuracy and diversity, where the diversity takes into account trivial symmetry transformations of the target network. We explain how this simple formulation generalizes variational inference. We use multi-layered perceptrons to form the mapping from the low dimensional input random vector to the high dimensional weight space, and demonstrate how to reduce the number of parameters in this mapping by parameter sharing. We perform experiments and show that the generated weights are diverse and lie on a non-trivial manifold.
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)
Cite as: arXiv:1801.01952 [stat.ML]
  (or arXiv:1801.01952v4 [stat.ML] for this version)

Submission history

From: Lior Deutsch [view email]
[v1] Sat, 6 Jan 2018 01:27:16 GMT (8795kb,D)
[v2] Sat, 17 Feb 2018 21:58:44 GMT (8793kb,D)
[v3] Thu, 5 Apr 2018 03:51:27 GMT (9035kb,D)
[v4] Fri, 6 Apr 2018 19:46:08 GMT (9035kb,D)

Link back to: arXiv, form interface, contact.