Current browse context:
cs.IT
Change to browse by:
References & Citations
Computer Science > Information Theory
Title: Minimax Rate-Optimal Estimation of Divergences between Discrete Distributions
(Submitted on 30 May 2016 (v1), last revised 3 Mar 2021 (this version, v5))
Abstract: We study the minimax estimation of $\alpha$-divergences between discrete distributions for integer $\alpha\ge 1$, which include the Kullback--Leibler divergence and the $\chi^2$-divergences as special examples. Dropping the usual theoretical tricks to acquire independence, we construct the first minimax rate-optimal estimator which does not require any Poissonization, sample splitting, or explicit construction of approximating polynomials. The estimator uses a hybrid approach which solves a problem-independent linear program based on moment matching in the non-smooth regime, and applies a problem-dependent bias-corrected plug-in estimator in the smooth regime, with a soft decision boundary between these regimes.
Submission history
From: Yanjun Han [view email][v1] Mon, 30 May 2016 07:24:03 GMT (223kb,D)
[v2] Thu, 24 Nov 2016 04:17:05 GMT (235kb,D)
[v3] Mon, 25 May 2020 11:02:28 GMT (44kb)
[v4] Thu, 29 Oct 2020 16:49:30 GMT (46kb)
[v5] Wed, 3 Mar 2021 06:36:52 GMT (46kb)
Link back to: arXiv, form interface, contact.