### Current browse context:

math.ST

### Change to browse by:

### References & Citations

# Mathematics > Statistics Theory

# Title: Phase transitions in nonparametric regressions: a curse of exploiting higher degree smoothness assumptions in finite samples

(Submitted on 7 Dec 2021 (v1), revised 4 Jul 2022 (this version, v4),

*latest version 8 Nov 2022*(v5))Abstract: When the regression function belongs to a smooth class consisting of univariate functions with derivatives up to the $(\gamma+1)$th order bounded in absolute values for a finite $\gamma$, it is generally viewed that exploiting higher degree smoothness assumptions helps reduce the estimation error. This paper shows that the minimax optimal mean integrated squared error (MISE) increases in $\gamma$ when the sample size $n$ is small relative to the order of $\left(\gamma+1\right)^{2\gamma+3}$, and decreases in $\gamma$ when $n$ is large relative to the order of $\left(\gamma+1\right)^{2\gamma+3}$. In particular, this phase transition property is shown to be achieved by common nonparametric procedures. Consider $\gamma_{1}$ and $\gamma_{2}$ such that $\gamma_{1}<\gamma_{2}$, where the $(\gamma_{2}+1)$th degree smoothness class is a subset of the $(\gamma_{1}+1)$th degree class. What is surprising about our results is that they imply, if $n$ is small relative to the order of $\left(\gamma_{1}+1\right)^{2\gamma_{1}+3}$, the optimal rate achieved by the estimator constrained to be in the smoother class (to exploit the $(\gamma_{2}+1)$th degree smoothness) is slower. In data sets with fewer than hundreds-of-thousands observations, our results suggest that one should not exploit beyond the third or fourth degree of smoothness. To some extent, our results provide a theoretical basis for the widely adopted practical recommendations given by Gelman and Imbens (2019).

The building blocks of our minimax optimality results are a set of metric entropy bounds we develop in this paper for smooth function classes. Some of our bounds are original, and some of them improve and/or generalize the ones in the literature.

## Submission history

From: Ying Zhu [view email]**[v1]**Tue, 7 Dec 2021 10:55:31 GMT (26kb)

**[v2]**Mon, 14 Feb 2022 17:41:20 GMT (27kb)

**[v3]**Thu, 16 Jun 2022 07:22:10 GMT (44kb)

**[v4]**Mon, 4 Jul 2022 10:23:09 GMT (47kb)

**[v5]**Tue, 8 Nov 2022 19:55:08 GMT (41kb)

Link back to: arXiv, form interface, contact.