We gratefully acknowledge support from
the Simons Foundation and member institutions.

Information Theory

New submissions

[ total of 18 entries: 1-18 ]
[ showing up to 2000 entries per page: fewer | more ]

New submissions for Fri, 15 Jan 21

[1]  arXiv:2101.05373 [pdf, other]
Title: On a Class of Time-Varying Gaussian ISI Channels
Authors: Kamyar Moshksar
Subjects: Information Theory (cs.IT)

This paper studies a class of stochastic and time-varying Gaussian intersymbol interference (ISI) channels. The $i^{th}$ channel tap during time slot $t$ is uniformly distributed over an interval of centre $c_i$ and radius $ r_{i}$. The array of channel taps is independent along both $t$ and $i$. The channel state information is unavailable at both the transmitter and the receiver. Lower and upper bounds are derived on the White-Gaussian-Input (WGI) capacity $C_{\scriptscriptstyle{WGI}}$ for arbitrary values of the radii $ r_i$. It is shown that $C_{\scriptscriptstyle{WGI}}$ does not scale with the average input power. The proposed lower bound is achieved by a joint-typicality decoder that is tuned to a set of candidates for the channel matrix. This set forms a net that covers the range of the random channel matrix and its resolution is optimized in order to yield the largest achievable rate. Tools in matrix analysis such as Weyl's inequality on perturbation of eigenvalues of symmetric matrices are used in order to analyze the probability of error.

[2]  arXiv:2101.05465 [pdf, ps, other]
Title: Noise Is Useful: Exploiting Data Diversity for Edge Intelligence
Comments: 5 pages, 6 figures, to be presented at IEEE Communications Letters
Subjects: Information Theory (cs.IT)

Edge intelligence requires to fast access distributed data samples generated by edge devices. The challenge is using limited radio resource to acquire massive data samples for training machine learning models at edge server. In this article, we propose a new communication-efficient edge intelligence scheme where the most useful data samples are selected to train the model. Here the usefulness or values of data samples is measured by data diversity which is defined as the difference between data samples. We derive a close-form expression of data diversity that combines data informativeness and channel quality. Then a joint data-and-channel diversity aware multiuser scheduling algorithm is proposed. We find that noise is useful for enhancing data diversity under some conditions.

[3]  arXiv:2101.05604 [pdf, ps, other]
Title: Decoding of Interleaved Linearized Reed-Solomon Codes with Applications to Network Coding
Comments: 6 pages, 2 figures, submitted to ISIT 2021
Subjects: Information Theory (cs.IT)

Recently, Martinez-Penas and Kschischang (IEEE Trans. Inf. Theory, 2019) showed that lifted linearized Reed-Solomon codes are suitable codes for error control in multi-shot network coding. We show how to construct and decode lifted interleaved linearized Reed-Solomon codes. Compared to the construction by Martinez-Penas--Kschischang, interleaving allows to increase the decoding region significantly (especially w.r.t. the number of insertions) and decreases the overhead due to the lifting (i.e., increases the code rate), at the cost of an increased packet size. The proposed decoder is a list decoder that can also be interpreted as a probabilistic unique decoder. Although our best upper bound on the list size is exponential, we present a heuristic argument and simulation results that indicate that the list size is in fact one for most channel realizations up to the maximal decoding radius.

[4]  arXiv:2101.05607 [pdf, other]
Title: Intelligent Reflecting Surfaces for Compute-and-Forward
Subjects: Information Theory (cs.IT); Networking and Internet Architecture (cs.NI)

In this paper, we show that using Intelligent Reflecting Surfaces (IRS) can enhance the computing capability of a wireless network scenario. We consider a Multiple Access Channel (MAC) where a number of users aim to send data to a Base Station (BS). The BS is interested in decoding a linear combination of the data from different users in the corresponding finite field. By focusing on the Compute-and-Forward framework, we show that through carefully choosing the IRS parameters, such a scenario's computation rate will be significantly improved. More specifically, we formulate an optimization problem to maximize the computation rate and tackle the problem via an alternating optimization (AO) approach. Our results confirm the usefulness of IRS technology for future wireless networks -- such as 6G -- with massive computation requirements.

[5]  arXiv:2101.05629 [pdf, ps, other]
Title: Off-grid Channel Estimation with Sparse Bayesian Learning for OTFS Systems
Comments: 30 pages, 9 figures, submitted to IEEE Transactions on Wireless Communications
Subjects: Information Theory (cs.IT)

This paper proposes an off-grid channel estimation scheme for orthogonal time-frequency space (OTFS) systems adopting the sparse Bayesian learning (SBL) framework. To avoid channel spreading caused by the fractional delay and Doppler shifts and to fully exploit the channel sparsity in the delay-Doppler (DD) domain, we estimate the original DD domain channel response rather than the effective DD domain channel response as commonly adopted in the literature. OTFS channel estimation is first formulated as a one-dimensional (1D) off-grid sparse signal recovery (SSR) problem based on a virtual sampling grid defined in the DD space, where the on-grid and off-grid components of the delay and Doppler shifts are separated for estimation. In particular, the on-grid components of the delay and Doppler shifts are jointly determined by the entry indices with significant values in the recovered sparse vector. Then, the corresponding off-grid components are modeled as hyper-parameters in the proposed SBL framework, which can be estimated via the expectation-maximization method. To strike a balance between channel estimation performance and computational complexity, we further propose a two-dimensional (2D) off-grid SSR problem via decoupling the delay and Doppler shift estimations. In our developed 1D and 2D off-grid SBL-based channel estimation algorithms, the hyper-parameters are updated alternatively for computing the conditional posterior distribution of channels, which can be exploited to reconstruct the effective DD domain channel. Compared with the 1D method, the proposed 2D method enjoys a much lower computational complexity while only suffers slight performance degradation. Simulation results verify the superior performance of the proposed channel estimation schemes over state-of-the-art schemes.

[6]  arXiv:2101.05681 [pdf, ps, other]
Title: Adaptive Private Distributed Matrix Multiplication
Comments: arXiv admin note: text overlap with arXiv:2004.12925
Subjects: Information Theory (cs.IT)

We consider the problem of designing codes with flexible rate (referred to as rateless codes), for private distributed matrix-matrix multiplication. A master server owns two private matrices $\mathbf{A}$ and $\mathbf{B}$ and hires worker nodes to help computing their multiplication. The matrices should remain information-theoretically private from the workers. Codes with fixed rate require the master to assign tasks to the workers and then wait for a predetermined number of workers to finish their assigned tasks. The size of the tasks, hence the rate of the scheme, depends on the number of workers that the master waits for. We design a rateless private matrix-matrix multiplication scheme, called RPM3. In contrast to fixed-rate schemes, our scheme fixes the size of the tasks and allows the master to send multiple tasks to the workers. The master keeps sending tasks and receiving results until it can decode the multiplication; rendering the scheme flexible and adaptive to heterogeneous environments. Despite resulting in a smaller rate than known straggler-tolerant schemes, RPM3 provides a smaller mean waiting time of the master by leveraging the heterogeneity of the workers. The waiting time is studied under two different models for the workers' service time. We provide upper bounds for the mean waiting time under both models. In addition, we provide lower bounds on the mean waiting time under the worker-dependent fixed service time model.

[7]  arXiv:2101.05792 [pdf, ps, other]
Title: Group Testing with a Graph Infection Spread Model
Subjects: Information Theory (cs.IT); Computers and Society (cs.CY); Data Structures and Algorithms (cs.DS); Networking and Internet Architecture (cs.NI); Signal Processing (eess.SP)

We propose a novel infection spread model based on a random connection graph which represents connections between $n$ individuals. Infection spreads via connections between individuals and this results in a probabilistic cluster formation structure as well as a non-i.i.d. (correlated) infection status for individuals. We propose a class of two-step sampled group testing algorithms where we exploit the known probabilistic infection spread model. We investigate the metrics associated with two-step sampled group testing algorithms. To demonstrate our results, for analytically tractable exponentially split cluster formation trees, we calculate the required number of tests and the expected number of false classifications in terms of the system parameters, and identify the trade-off between them. For such exponentially split cluster formation trees, for zero-error construction, we prove that the required number of tests is $O(\log_2n)$. Thus, for such cluster formation trees, our algorithm outperforms any zero-error non-adaptive group test, binary splitting algorithm, and Hwang's generalized binary splitting algorithm. Our results imply that, by exploiting probabilistic information on the connections of individuals, group testing can be used to reduce the number of required tests significantly even when infection rate is high, contrasting the prevalent belief that group testing is useful only when infection rate is low.

Cross-lists for Fri, 15 Jan 21

[8]  arXiv:2009.01678 (cross-list from math.PR) [pdf, ps, other]
Title: Hamilton-Jacobi equations for inference of matrix tensor products
Comments: 44 pages
Subjects: Probability (math.PR); Information Theory (cs.IT)

We study the high-dimensional limit of the free energy associated with the inference problem of finite-rank matrix tensor products. In general, we bound the limit from above by the unique solution to a certain Hamilton-Jacobi equation. Under additional assumptions on the nonlinearity in the equation which is determined explicitly by the model, we identify the limit with the solution. Two notions of solutions, weak solutions and viscosity solutions, are considered, each of which has its own advantages and requires different treatments. For concreteness, we apply our results to a model with i.i.d. entries and symmetric interactions. In particular, for the first order and even order tensor products, we identify the limit and obtain estimates on convergence rates; for other odd orders, upper bounds are obtained.

[9]  arXiv:2101.05544 (cross-list from cs.LG) [pdf, other]
Title: DICE: Diversity in Deep Ensembles via Conditional Redundancy Adversarial Estimation
Comments: Published as a conference paper at ICLR 2021. 9 main pages, 13 figures, 12 tables
Subjects: Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV); Information Theory (cs.IT)

Deep ensembles perform better than a single network thanks to the diversity among their members. Recent approaches regularize predictions to increase diversity; however, they also drastically decrease individual members' performances. In this paper, we argue that learning strategies for deep ensembles need to tackle the trade-off between ensemble diversity and individual accuracies. Motivated by arguments from information theory and leveraging recent advances in neural estimation of conditional mutual information, we introduce a novel training criterion called DICE: it increases diversity by reducing spurious correlations among features. The main idea is that features extracted from pairs of members should only share information useful for target class prediction without being conditionally redundant. Therefore, besides the classification loss with information bottleneck, we adversarially prevent features from being conditionally predictable from each other. We manage to reduce simultaneous errors while protecting class information. We obtain state-of-the-art accuracy results on CIFAR-10/100: for example, an ensemble of 5 networks trained with DICE matches an ensemble of 7 networks trained independently. We further analyze the consequences on calibration, uncertainty estimation, out-of-distribution detection and online co-distillation.

Replacements for Fri, 15 Jan 21

[10]  arXiv:1702.07409 (replaced) [pdf, other]
Title: Founsure 1.0: An Erasure Code Library with Efficient Repair and Update Features
Comments: Accepted to Elsevier SoftwareX, 2021
Subjects: Information Theory (cs.IT)
[11]  arXiv:1712.01497 (replaced) [pdf, other]
Title: Atomic Norm Based Localization of Far-Field and Near-Field Signals with Generalized Symmetric Arrays
Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)
[12]  arXiv:1908.02503 (replaced) [pdf, other]
Title: Computational approaches to non-convex, sparsity-inducing multi-penalty regularization
Comments: 20 pages, 2 figures
Subjects: Information Theory (cs.IT)
[13]  arXiv:1912.11662 (replaced) [pdf, other]
Title: Terahertz Multi-User Massive MIMO with Intelligent Reflecting Surface: Beam Training and Hybrid Beamforming
Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)
[14]  arXiv:2006.04581 (replaced) [pdf, other]
Title: Precoder Design and Power Allocation for Downlink MIMO-NOMA via Simultaneous Triangularization
Comments: Accepted for presentation at the 2021 IEEE Wireless Communications and Networking Conference. This paper is the conference version of arXiv:2006.06471 with 6pp, 2 figures, for code, see this https URL
Subjects: Information Theory (cs.IT)
[15]  arXiv:2006.06471 (replaced) [pdf, other]
Title: Uplink and Downlink MIMO-NOMA with Simultaneous Triangularization
Comments: Accepted by the IEEE Transactions on Wireless Communications. This is the journal version of the submission arXiv:2006.04581 with 33 pages, 10 figures, and 2 tables. For associated code see this https URL
Journal-ref: IEEE Transactions on Wireless Communications, 2021
Subjects: Information Theory (cs.IT)
[16]  arXiv:2011.14741 (replaced) [pdf, ps, other]
Title: Minimax Converse for Identification via Channels
Authors: Shun Watanabe
Comments: 18 pages, no figure
Subjects: Information Theory (cs.IT)
[17]  arXiv:2101.03225 (replaced) [pdf, ps, other]
Title: The extended binary quadratic residue code of length 42 holds a 3-design
Comments: 6 pages. Second version
Subjects: Information Theory (cs.IT); Combinatorics (math.CO)
[18]  arXiv:2007.08158 (replaced) [pdf, other]
Title: Channel Estimation for RIS-Aided mmWave MIMO Systems via Atomic Norm Minimization
Comments: 30 pages, 10 figures, Submitted to IEEE TWC, under second round review
Subjects: Signal Processing (eess.SP); Information Theory (cs.IT)
[ total of 18 entries: 1-18 ]
[ showing up to 2000 entries per page: fewer | more ]

Disable MathJax (What is MathJax?)

Links to: arXiv, form interface, find, cs, recent, 2101, contact, help  (Access key information)