Current browse context:
stat.ML
Change to browse by:
References & Citations
Statistics > Machine Learning
Title: Double Descent Risk and Volume Saturation Effects: A Geometric Perspective
(Submitted on 8 Jun 2020 (v1), last revised 10 Nov 2020 (this version, v2))
Abstract: The appearance of the double-descent risk phenomenon has received growing interest in the machine learning and statistics community, as it challenges well-understood notions behind the U-shaped train-test curves. Motivated through Rissanen's minimum description length (MDL), Balasubramanian's Occam's Razor, and Amari's information geometry, we investigate how the logarithm of the model volume: $\log V$, works to extend intuition behind the AIC and BIC model selection criteria. We find that for the particular model classes of isotropic linear regression and statistical lattices, the $\log V$ term may be decomposed into a sum of distinct components, each of which assist in their explanations of the appearance of this phenomenon. In particular they suggest why generalization error does not necessarily continue to grow with increasing model dimensionality.
Submission history
From: Prasad Cheema [view email][v1] Mon, 8 Jun 2020 05:47:57 GMT (3922kb,D)
[v2] Tue, 10 Nov 2020 05:18:20 GMT (3959kb,D)
Link back to: arXiv, form interface, contact.