We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

math.PR

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Probability

Title: On Optimal Uniform Concentration Inequalities for Discrete Entropies in the High-dimensional Setting

Authors: Yunpeng Zhao
Abstract: We prove an exponential decay concentration inequality to bound the tail probability of the difference between the log-likelihood of discrete random variables and the negative entropy. The concentration bound we derive holds uniformly over all parameter values. The new result improves the convergence rate in an earlier result of Zhao (2020), from $(K^2\log K)/n=o(1)$ to $ (\log K)^2/n=o(1)$, where $n$ is the sample size and $K$ is the number of possible values of the discrete variable. We further prove that the rate $(\log K)^2/n=o(1)$ is optimal. The results are extended to misspecified log-likelihoods for grouped random variables.
Subjects: Probability (math.PR); Information Theory (cs.IT); Statistics Theory (math.ST)
MSC classes: 60E15
Cite as: arXiv:2007.04547 [math.PR]
  (or arXiv:2007.04547v2 [math.PR] for this version)

Submission history

From: Yunpeng Zhao [view email]
[v1] Thu, 9 Jul 2020 04:20:38 GMT (13kb)
[v2] Mon, 27 Jul 2020 00:30:52 GMT (13kb)

Link back to: arXiv, form interface, contact.