Current browse context:
cs.IT
Change to browse by:
References & Citations
Computer Science > Information Theory
Title: An Effective Bernstein-type Bound on Shannon Entropy over Countably Infinite Alphabets
(Submitted on 23 Jun 2021)
Abstract: We prove a Bernstein-type bound for the difference between the average of negative log-likelihoods of independent discrete random variables and the Shannon entropy, both defined on a countably infinite alphabet. The result holds for the class of discrete random variables with tails lighter than or on the same order of a discrete power-law distribution. Most commonly-used discrete distributions such as the Poisson distribution, the negative binomial distribution, and the power-law distribution itself belong to this class. The bound is effective in the sense that we provide a method to compute the constants in it.
Link back to: arXiv, form interface, contact.