We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Information Theory

Title: Minimax Rate-Optimal Estimation of Divergences between Discrete Distributions

Abstract: We study the minimax estimation of $\alpha$-divergences between discrete distributions for integer $\alpha\ge 1$, which include the Kullback--Leibler divergence and the $\chi^2$-divergences as special examples. Dropping the usual theoretical tricks to acquire independence, we construct the first minimax rate-optimal estimator which does not require any Poissonization, sample splitting, or explicit construction of approximating polynomials. The estimator uses a hybrid approach which solves a problem-independent linear program based on moment matching in the non-smooth regime, and applies a problem-dependent bias-corrected plug-in estimator in the smooth regime, with a soft decision boundary between these regimes.
Comments: This (v5) is a significantly revised version of (v2), and fixed some typos in (v4)
Subjects: Information Theory (cs.IT); Statistics Theory (math.ST)
Journal reference: Published in IEEE Journal on Selected Areas in Information Theory, vol. 1, no. 3, pp. 814-823, Nov. 2020
DOI: 10.1109/JSAIT.2020.3041036
Cite as: arXiv:1605.09124 [cs.IT]
  (or arXiv:1605.09124v5 [cs.IT] for this version)

Submission history

From: Yanjun Han [view email]
[v1] Mon, 30 May 2016 07:24:03 GMT (223kb,D)
[v2] Thu, 24 Nov 2016 04:17:05 GMT (235kb,D)
[v3] Mon, 25 May 2020 11:02:28 GMT (44kb)
[v4] Thu, 29 Oct 2020 16:49:30 GMT (46kb)
[v5] Wed, 3 Mar 2021 06:36:52 GMT (46kb)

Link back to: arXiv, form interface, contact.