We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ML

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Machine Learning

Title: Deep Maxout Network Gaussian Process

Abstract: Study of neural networks with infinite width is important for better understanding of the neural network in practical application. In this work, we derive the equivalence of the deep, infinite-width maxout network and the Gaussian process (GP) and characterize the maxout kernel with a compositional structure. Moreover, we build up the connection between our deep maxout network kernel and deep neural network kernels. We also give an efficient numerical implementation of our kernel which can be adapted to any maxout rank. Numerical results show that doing Bayesian inference based on the deep maxout network kernel can lead to competitive results compared with their finite-width counterparts and deep neural network kernels. This enlightens us that the maxout activation may also be incorporated into other infinite-width neural network structures such as the convolutional neural network (CNN).
Comments: 15 pages, 3 figures
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)
ACM classes: G.3
Cite as: arXiv:2208.04468 [stat.ML]
  (or arXiv:2208.04468v1 [stat.ML] for this version)

Submission history

From: Libin Liang [view email]
[v1] Mon, 8 Aug 2022 23:52:26 GMT (617kb,D)
[v2] Fri, 26 Aug 2022 15:40:00 GMT (617kb,D)

Link back to: arXiv, form interface, contact.