We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.IT

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Information Theory

Title: On the Renyi Divergence, Joint Range of Relative Entropies, and a Channel Coding Theorem

Authors: Igal Sason
Abstract: This paper starts by considering the minimization of the Renyi divergence subject to a constraint on the total variation distance. Based on the solution of this optimization problem, the exact locus of the points $\bigl( D(Q\|P_1), D(Q\|P_2) \bigr)$ is determined when $P_1, P_2, Q$ are arbitrary probability measures which are mutually absolutely continuous, and the total variation distance between $P_1$ and $P_2$ is not below a given value. It is further shown that all the points of this convex region are attained by probability measures which are defined on a binary alphabet. This characterization yields a geometric interpretation of the minimal Chernoff information subject to a constraint on the total variation distance. This paper also derives an exponential upper bound on the performance of binary linear block codes (or code ensembles) under maximum-likelihood decoding. Its derivation relies on the Gallager bounding technique, and it reproduces the Shulman-Feder bound as a special case. The bound is expressed in terms of the Renyi divergence from the normalized distance spectrum of the code (or the average distance spectrum of the ensemble) to the binomially distributed distance spectrum of the capacity-achieving ensemble of random block codes. This exponential bound provides a quantitative measure of the degradation in performance of binary linear block codes (or code ensembles) as a function of the deviation of their distance spectra from the binomial distribution. An efficient use of this bound is considered.
Comments: Submitted to IEEE Trans. on Information Theory in March 2015; revised in October 2015
Subjects: Information Theory (cs.IT); Probability (math.PR)
Cite as: arXiv:1501.03616 [cs.IT]
  (or arXiv:1501.03616v4 [cs.IT] for this version)

Submission history

From: Igal Sason [view email]
[v1] Thu, 15 Jan 2015 10:03:57 GMT (86kb)
[v2] Wed, 18 Feb 2015 07:22:27 GMT (87kb)
[v3] Wed, 4 Mar 2015 00:15:01 GMT (91kb)
[v4] Mon, 26 Oct 2015 13:52:57 GMT (90kb)

Link back to: arXiv, form interface, contact.