Current browse context:
math.ST
Change to browse by:
References & Citations
Mathematics > Statistics Theory
Title: Quantile regression with deep ReLU Networks: Estimators and minimax rates
(Submitted on 16 Oct 2020 (v1), last revised 18 Dec 2020 (this version, v5))
Abstract: Quantile regression is the task of estimating a specified percentile response, such as the median, from a collection of known covariates. We study quantile regression with rectified linear unit (ReLU) neural networks as the chosen model class. We derive an upper bound on the expected mean squared error of a ReLU network used to estimate any quantile conditional on a set of covariates. This upper bound only depends on the best possible approximation error, the number of layers in the network, and the number of nodes per layer. We further show upper bounds that are tight for two large classes of functions: compositions of H\"older functions and members of a Besov space. These tight bounds imply ReLU networks with quantile regression achieve minimax rates for broad collections of function types. Unlike existing work, the theoretical results hold under minimal assumptions and apply to general error distributions, including heavy-tailed distributions. Empirical simulations on a suite of synthetic response functions demonstrate the theoretical results translate to practical implementations of ReLU networks. Overall, the theoretical and empirical results provide insight into the strong performance of ReLU neural networks for quantile regression across a broad range of function classes and error distributions. All code for this paper is publicly available at this https URL
Submission history
From: Oscar Hernan Madrid Padilla [view email][v1] Fri, 16 Oct 2020 08:34:04 GMT (31458kb,D)
[v2] Mon, 19 Oct 2020 10:20:07 GMT (31459kb,D)
[v3] Sun, 25 Oct 2020 11:39:50 GMT (31460kb,D)
[v4] Tue, 27 Oct 2020 04:53:49 GMT (31460kb,D)
[v5] Fri, 18 Dec 2020 02:40:16 GMT (31460kb,D)
Link back to: arXiv, form interface, contact.