We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ML

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Machine Learning

Title: Double Descent Risk and Volume Saturation Effects: A Geometric Perspective

Abstract: The appearance of the double-descent risk phenomenon has received growing interest in the machine learning and statistics community, as it challenges well-understood notions behind the U-shaped train-test curves. Motivated through Rissanen's minimum description length (MDL), Balasubramanian's Occam's Razor, and Amari's information geometry, we investigate how the logarithm of the model volume: $\log V$, works to extend intuition behind the AIC and BIC model selection criteria. We find that for the particular model classes of isotropic linear regression and statistical lattices, the $\log V$ term may be decomposed into a sum of distinct components, each of which assist in their explanations of the appearance of this phenomenon. In particular they suggest why generalization error does not necessarily continue to grow with increasing model dimensionality.
Comments: Updated version. Some parts have been re-structured, and certain elements shifted to Appendix
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)
Cite as: arXiv:2006.04366 [stat.ML]
  (or arXiv:2006.04366v2 [stat.ML] for this version)

Submission history

From: Prasad Cheema [view email]
[v1] Mon, 8 Jun 2020 05:47:57 GMT (3922kb,D)
[v2] Tue, 10 Nov 2020 05:18:20 GMT (3959kb,D)

Link back to: arXiv, form interface, contact.