We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.IT

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Information Theory

Title: Bounds on the Entropy of a Function of a Random Variable and their Applications

Abstract: It is well known that the entropy $H(X)$ of a discrete random variable $X$ is always greater than or equal to the entropy $H(f(X))$ of a function $f$ of $X$, with equality if and only if $f$ is one-to-one. In this paper, we give tight bounds on $H(f(X))$ when the function $f$ is not one-to-one, and we illustrate a few scenarios where this matters. As an intermediate step towards our main result, we derive a lower bound on the entropy of a probability distribution, when only a bound on the ratio between the maximal and minimal probabilities is known. The lower bound improves on previous results in the literature, and it could find applications outside the present scenario.
Comments: Accepted for publications to IEEE Transactions on Information Theory
Subjects: Information Theory (cs.IT); Data Structures and Algorithms (cs.DS)
Cite as: arXiv:1712.07906 [cs.IT]
  (or arXiv:1712.07906v1 [cs.IT] for this version)

Submission history

From: Ugo Vaccaro [view email]
[v1] Thu, 21 Dec 2017 12:28:48 GMT (188kb,D)

Link back to: arXiv, form interface, contact.