References & Citations
Computer Science > Information Theory
Title: Information Structures of Extremum Problems of Feedback Capacity for General Channels with Memory
(Submitted on 4 Apr 2016 (this version), latest version 19 Aug 2016 (v2))
Abstract: For any class of channel conditional distributions, with finite memory dependence on channel input RVs $A^n {\stackrel{\triangle}{=}} \{A_i: i=0, \ldots, n\}$ or channel output RVs $B^n {\stackrel{\triangle}{=}} \{B_i: i=0, \ldots, n\}$ or both, we characterize the subsets of channel input distributions ${\cal P}_{[0,n]}^{CI}\subseteq {\cal P}_{[0,n]} {\stackrel{\triangle}{=}} \big\{ {\bf P}_{A_i|A^{i-1}, B^{i-1}}: i=1, \ldots, n\big\}$, which satisfy conditional independence on past information, and maximize directed information, defined by $I(A^n \rightarrow B^n)$. We derive the corresponding expressions, called "characterizations of Finite Transmission Feedback Information (FTFI) capacity". We derive similar characterizations, when general transmission cost constraints are imposed. Moreover, we also show that the structural properties apply to general nonlinear and linear autoregressive channel models defined by discrete-time recursions on general alphabet spaces, and driven by arbitrary distributed noise processes. We derive these structural properties by invoking stochastic optimal control theory and variational equalities of directed information, to identify tight upper bounds on $I(A^n \rightarrow B^n)$, which are achievable over subsets of conditional independence distributions ${\cal P}_{[0,n]}^{CI} \subseteq {\cal P}_{[0,n]}$ and specified by the dependence of channel distributions and transmission cost functions on inputs and output symbols. We apply the characterizations to recursive Multiple Input Multiple Output Gaussian Linear Channel Models with limited memory on channel input and output sequences. The structural properties of optimal channel input distributions, generalize the structural properties of Memoryless Channels with feedback, to any channel distribution with memory, and settle various long standing problems in information theory.
Submission history
From: Christos Kourtellaris [view email][v1] Mon, 4 Apr 2016 21:05:02 GMT (86kb,D)
[v2] Fri, 19 Aug 2016 14:37:23 GMT (111kb,D)
Link back to: arXiv, form interface, contact.