### Current browse context:

math.PR

### Change to browse by:

### References & Citations

# Mathematics > Probability

# Title: On Optimal Uniform Concentration Inequalities for Discrete Entropies in the High-dimensional Setting

(Submitted on 9 Jul 2020 (v1), last revised 27 Jul 2020 (this version, v2))

Abstract: We prove an exponential decay concentration inequality to bound the tail probability of the difference between the log-likelihood of discrete random variables and the negative entropy. The concentration bound we derive holds uniformly over all parameter values. The new result improves the convergence rate in an earlier result of Zhao (2020), from $(K^2\log K)/n=o(1)$ to $ (\log K)^2/n=o(1)$, where $n$ is the sample size and $K$ is the number of possible values of the discrete variable. We further prove that the rate $(\log K)^2/n=o(1)$ is optimal. The results are extended to misspecified log-likelihoods for grouped random variables.

## Submission history

From: Yunpeng Zhao [view email]**[v1]**Thu, 9 Jul 2020 04:20:38 GMT (13kb)

**[v2]**Mon, 27 Jul 2020 00:30:52 GMT (13kb)

Link back to: arXiv, form interface, contact.