References & Citations
Computer Science > Information Theory
Title: On the Kullback-Leibler divergence between discrete normal distributions
(Submitted on 30 Sep 2021 (v1), last revised 18 Oct 2021 (this version, v3))
Abstract: Discrete normal distributions are defined as the distributions with prescribed means and covariance matrices which maximize entropy on the integer lattice support. The set of discrete normal distributions form an exponential family with cumulant function related to the Riemann theta function. In this paper, we present several formula for common statistical divergences between discrete normal distributions including the Kullback-Leibler divergence. In particular, we describe an efficient approximation technique for calculating the Kullback-Leibler divergence between discrete normal distributions via the R\'enyi $\alpha$-divergences or the projective $\gamma$-divergences.
Submission history
From: Frank Nielsen [view email][v1] Thu, 30 Sep 2021 08:39:49 GMT (150kb,D)
[v2] Thu, 7 Oct 2021 08:02:54 GMT (228kb,D)
[v3] Mon, 18 Oct 2021 04:07:58 GMT (229kb,D)
Link back to: arXiv, form interface, contact.