We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.LG

Change to browse by:

cs

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: NeuralEF: Deconstructing Kernels by Deep Neural Networks

Abstract: Learning the principal eigenfunctions of an integral operator defined by a kernel and a data distribution is at the core of many machine learning problems. Traditional nonparametric solutions based on the Nystr{\"o}m formula suffer from scalability issues. Recent work has resorted to a parametric approach, i.e., training neural networks to approximate the eigenfunctions. However, the existing method relies on an expensive orthogonalization step and is difficult to implement. We show that these problems can be fixed by using a new series of objective functions that generalizes the EigenGame~\citep{gemp2020eigengame} to function space. We test our method on a variety of supervised and unsupervised learning problems and show it provides accurate approximations to the eigenfunctions of polynomial, radial basis, neural network Gaussian process, and neural tangent kernels. Finally, we demonstrate our method can scale up linearised Laplace approximation of deep neural networks to modern image classification datasets through approximating the Gauss-Newton matrix. Code is available at \url{this https URL}.
Comments: International Conference on Machine Learning (ICML), 2022
Subjects: Machine Learning (cs.LG)
Cite as: arXiv:2205.00165 [cs.LG]
  (or arXiv:2205.00165v4 [cs.LG] for this version)

Submission history

From: Zhijie Deng [view email]
[v1] Sat, 30 Apr 2022 05:31:07 GMT (2756kb,D)
[v2] Mon, 13 Jun 2022 03:03:16 GMT (4526kb,D)
[v3] Fri, 17 Jun 2022 12:26:42 GMT (4525kb,D)
[v4] Sun, 23 Oct 2022 07:23:14 GMT (2140kb,D)

Link back to: arXiv, form interface, contact.