We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.IT

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Information Theory

Title: Fisher Information and Mutual Information Constraints

Abstract: We consider the processing of statistical samples $X\sim P_\theta$ by a channel $p(y|x)$, and characterize how the statistical information from the samples for estimating the parameter $\theta\in\mathbb{R}^d$ can scale with the mutual information or capacity of the channel. We show that if the statistical model has a sub-Gaussian score function, then the trace of the Fisher information matrix for estimating $\theta$ from $Y$ can scale at most linearly with the mutual information between $X$ and $Y$. We apply this result to obtain minimax lower bounds in distributed statistical estimation problems, and obtain a tight preconstant for Gaussian mean estimation. We then show how our Fisher information bound can also imply mutual information or Jensen-Shannon divergence based distributed strong data processing inequalities.
Subjects: Information Theory (cs.IT); Statistics Theory (math.ST)
Cite as: arXiv:2102.05802 [cs.IT]
  (or arXiv:2102.05802v2 [cs.IT] for this version)

Submission history

From: Leighton Barnes [view email]
[v1] Thu, 11 Feb 2021 01:53:09 GMT (11kb)
[v2] Thu, 8 Jul 2021 22:28:26 GMT (12kb)

Link back to: arXiv, form interface, contact.