We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:


Current browse context:


Change to browse by:

References & Citations

DBLP - CS Bibliography


(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Information Theory

Title: Information Structures of Maximizing Distributions of Feedback Capacity for General Channels with Memory & Applications

Abstract: For any class of channel conditional distributions, with finite memory dependence on channel input RVs $A^n {\stackrel{\triangle}{=}} \{A_i: i=0, \ldots, n\}$ or channel output RVs $B^n {\stackrel{\triangle}{=}} \{B_i: i=0, \ldots, n\}$ or both, we characterize the sets of channel input distributions, which maximize directed information defined by $ I(A^n \rightarrow B^n) {\stackrel{\triangle}{=}} \sum_{i=0}^n I(A^i;B_i|B^{i-1}) $ and we derive the corresponding expressions, called "characterizations of Finite Transmission Feedback Information (FTFI) capacity". The main theorems state that optimal channel input distributions occur in subsets ${\cal P}_{[0,n]}^{CI}\subseteq {\cal P}_{[0,n]} {\stackrel{\triangle}{=}}\big\{ {\bf P}_{A_i|A^{i-1}, B^{i-1}}: i=0, \ldots, n\big\}$, which satisfy conditional independence on past information. We derive similar characterizations, when general transmission cost constraints are imposed. Moreover, we also show that the structural properties apply to general nonlinear and linear autoregressive channel models defined by discrete-time recursions on general alphabet spaces, and driven by arbitrary distributed noise processes. We derive these structural properties by invoking stochastic optimal control theory and variational equalities of directed information, to identify tight upper bounds on $I(A^n \rightarrow B^n)$, which are achievable over subsets of conditional distributions ${\cal P}_{[0,n]}^{CI} \subseteq {\cal P}_{[0,n]}$, which satisfy conditional independence and they are specified by the dependence of channel distributions and transmission cost functions on inputs and output symbols. We apply the characterizations to recursive Multiple Input Multiple Output Gaussian Linear Channel Models with limited memory and we show a separation principle between the computation of the elements of the optimal strategies.
Comments: Submitted to the IEEE Transactions on Information Theory,Feb. 2016, paper IT-16-0120
Subjects: Information Theory (cs.IT)
Cite as: arXiv:1604.01063 [cs.IT]
  (or arXiv:1604.01063v2 [cs.IT] for this version)

Submission history

From: Christos Kourtellaris [view email]
[v1] Mon, 4 Apr 2016 21:05:02 GMT (86kb,D)
[v2] Fri, 19 Aug 2016 14:37:23 GMT (111kb,D)

Link back to: arXiv, form interface, contact.