We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ML

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Machine Learning

Title: Provably Strict Generalisation Benefit for Invariance in Kernel Methods

Authors: Bryn Elesedy
Abstract: It is a commonly held belief that enforcing invariance improves generalisation. Although this approach enjoys widespread popularity, it is only very recently that a rigorous theoretical demonstration of this benefit has been established. In this work we build on the function space perspective of Elesedy and Zaidi arXiv:2102.10333 to derive a strictly non-zero generalisation benefit of incorporating invariance in kernel ridge regression when the target is invariant to the action of a compact group. We study invariance enforced by feature averaging and find that generalisation is governed by a notion of effective dimension that arises from the interplay between the kernel and the group. In building towards this result, we find that the action of the group induces an orthogonal decomposition of both the reproducing kernel Hilbert space and its kernel, which may be of interest in its own right.
Comments: Appeared in NeurIPS 2021
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)
Cite as: arXiv:2106.02346 [stat.ML]
  (or arXiv:2106.02346v2 [stat.ML] for this version)

Submission history

From: Bryn Elesedy [view email]
[v1] Fri, 4 Jun 2021 08:55:28 GMT (30kb)
[v2] Sun, 19 Dec 2021 22:40:15 GMT (32kb)

Link back to: arXiv, form interface, contact.