Title: On the Rényi Divergence and the Joint Range of Relative Entropies

Abstract: This paper starts with a study of the minimum of the R\'{e}nyi divergence subject to a fixed (or minimal) value of the total variation distance. Relying on the solution of this minimization problem, we determine the exact region of the points $\bigl( D(Q||P_1), D(Q||P_2) \bigr)$ where $P_1$ and $P_2$ are any probability distributions whose total variation distance is not below a fixed value, and the probability distribution $Q$ is arbitrary (none of these three distributions is assumed to be fixed). It is further shown that all the points of this convex region are attained by a triple of 2-element probability distributions. As a byproduct of this characterization, we provide a geometric interpretation of the minimal Chernoff information subject to a minimal total variation distance. Finally, as a separate issue, a new exponential upper bound on the block error probability of ML decoded binary linear block codes is introduced. Its error exponent depends on the R\'{e}nyi divergence (of orders above 1), used for quantifying the degradation in performance of block codes in terms of the deviation of their distance spectra from the binomial distribution.