We gratefully acknowledge support from
the Simons Foundation and member institutions.

Information Theory

New submissions

[ total of 24 entries: 1-24 ]
[ showing up to 2000 entries per page: fewer | more ]

New submissions for Thu, 25 Nov 21

[1]  arXiv:2111.12166 [pdf, other]
Title: Towards Empirical Sandwich Bounds on the Rate-Distortion Function
Subjects: Information Theory (cs.IT); Machine Learning (cs.LG); Machine Learning (stat.ML)

Rate-distortion (R-D) function, a key quantity in information theory, characterizes the fundamental limit of how much a data source can be compressed subject to a fidelity criterion, by any compression algorithm. As researchers push for ever-improving compression performance, establishing the R-D function of a given data source is not only of scientific interest, but also sheds light on the possible room for improving compression algorithms. Previous work on this problem relied on distributional assumptions on the data source (Gibson, 2017) or only applied to discrete data. By contrast, this paper makes the first attempt at an algorithm for sandwiching the R-D function of a general (not necessarily discrete) source requiring only i.i.d. data samples. We estimate R-D sandwich bounds on Gaussian and high-dimension banana-shaped sources, as well as GAN-generated images. Our R-D upper bound on natural images indicates room for improving the performance of state-of-the-art image compression methods by 1 dB in PSNR at various bitrates.

[2]  arXiv:2111.12181 [pdf, other]
Title: Channel Characterization of Diffusion-based Molecular Communication with Multiple Fully-absorbing Receivers
Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)

In this paper an analytical model is introduced to describe the impulse response of the diffusive channel between a pointwise transmitter and a given fully-absorbing (FA) receiver in a molecular communication (MC) system. The presence of neighbouring FA nanomachines in the environment is taken into account by describing them as sources of negative molecules. The channel impulse responses of all the receivers are linked in a system of integral equations. The solution of the system with two receivers is obtained analytically. For a higher number of receivers the system of integral equations is solved numerically. It is also shown that the channel impulse response shape is distorted by the presence of the interferers. For instance, there is a time shift of the peak in the number of absorbed molecules compared to the case without interference, as predicted by the proposed model. The analytical derivations are validated by means of particle based simulations.

[3]  arXiv:2111.12212 [pdf, other]
Title: Long-Term CSI-based Design for RIS-Aided Multiuser MISO Systems Exploiting Deep Reinforcement Learning
Comments: Under revision in IEEE journal. Keywords: Reconfigurable intelligent surface (RIS), intelligent reflecting surface (IRS)
Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)

In this paper, we study the transmission design for reconfigurable intelligent surface (RIS)-aided multiuser communication networks. Different from most of the existing contributions, we consider long-term CSI-based transmission design, where both the beamforming vectors at the base station (BS) and the phase shifts at the RIS are designed based on long-term CSI, which can significantly reduce the channel estimation overhead. Due to the lack of explicit ergodic data rate expression, we propose a novel deep deterministic policy gradient (DDPG) based algorithm to solve the optimization problem, which was trained by using the channel vectors generated in an offline manner. Simulation results demonstrate that the achievable net throughput is higher than that achieved by the conventional instantaneous-CSI based scheme when taking the channel estimation overhead into account.

[4]  arXiv:2111.12282 [pdf, ps, other]
Title: Self-orthogonality matrix and Reed-Muller code
Subjects: Information Theory (cs.IT)

Kim et al. (2021) gave a method to embed a given binary $[n,k]$ code $\mathcal{C}$ $(k = 3, 4)$ into a self-orthogonal code of the shortest length which has the same dimension $k$ and minimum distance $d' \ge d(\mathcal{C})$. We extends this result for $k=5$ and $6$ by proposing a new method related to a special matrix, called the self-orthogonality matrix $SO_k$, obtained by shortnening a Reed-Muller code $\mathcal{R}(2,k)$. Furthermore, we disprove partially the conjecture (Kim et al. (2021)) by showing that if $31 \le n \le 256$ and $n\equiv 14,22,29 \pmod{31}$, then there exist optimal $[n,5]$ codes which are self-orthogonal. We also construct optimal self-orthogonal $[n,6]$ codes when $41 \le n \le 256$ satisfies $n \ne 46, 54, 61$ and $n \not\equiv 7, 14, 22, 29, 38, 45, 53, 60 \pmod{63}$.

[5]  arXiv:2111.12382 [pdf, ps, other]
Title: Compressed Sensing Channel Estimation for OTFS Modulation in Non-Integer Delay-Doppler Domain
Comments: This is the author's self-archive preprint of a paper accepted in IEEE GLOBECOM 2021
Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)

This paper introduces a Compressed Sensing (CS) estimation scheme for Orthogonal Time Frequency Space (OTFS) channels with sparse multipath. The OTFS waveform represents signals in a two dimensional Delay-Doppler (DD) orthonormal basis. The proposed model does not require the assumption that the delays are integer multiples of the sampling period. The analysis shows that non-integer delay and Doppler shifts in the channel cannot be accurately modelled by integer approximations. An Orthogonal Matching Pursuit with Binary-division Refinement (OMPBR) estimation algorithm is proposed. The proposed estimator finds the best channel approximation over a continuous DD dictionary without integer approximations. This results in a significant reduction of the estimation normalized mean squared error with reasonable computational complexity.

[6]  arXiv:2111.12444 [pdf, other]
Title: Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applications
Comments: This work is a JSAC invited survey & tutorial paper. Copyright may be transferred without notice, after which this version may no longer be accessible
Subjects: Information Theory (cs.IT); Machine Learning (cs.LG); Networking and Internet Architecture (cs.NI); Signal Processing (eess.SP)

The thriving of artificial intelligence (AI) applications is driving the further evolution of wireless networks. It has been envisioned that 6G will be transformative and will revolutionize the evolution of wireless from "connected things" to "connected intelligence". However, state-of-the-art deep learning and big data analytics based AI systems require tremendous computation and communication resources, causing significant latency, energy consumption, network congestion, and privacy leakage in both of the training and inference processes. By embedding model training and inference capabilities into the network edge, edge AI stands out as a disruptive technology for 6G to seamlessly integrate sensing, communication, computation, and intelligence, thereby improving the efficiency, effectiveness, privacy, and security of 6G networks. In this paper, we shall provide our vision for scalable and trustworthy edge AI systems with integrated design of wireless communication strategies and decentralized machine learning models. New design principles of wireless networks, service-driven resource allocation optimization methods, as well as a holistic end-to-end system architecture to support edge AI will be described. Standardization, software and hardware platforms, and application scenarios are also discussed to facilitate the industrialization and commercialization of edge AI systems.

[7]  arXiv:2111.12489 [pdf, ps, other]
Title: Repeated-root Constacyclic Codes with Optimal Locality
Subjects: Information Theory (cs.IT)

A code is called a locally repairable code (LRC) if any code symbol is a function of a small fraction of other code symbols. When a locally repairable code is employed in a distributed storage systems, an erased symbol can be recovered by accessing only a small number of other symbols, and hence alleviating the network resources required during the repair process. In this paper we consider repeated-root constacyclic codes, which is a generalization of cyclic codes, that are optimal with respect to a Singleton-like bound on minimum distance. An LRC with the structure of a constacyclic code can be encoded efficiently using any encoding algorithm for constacyclic codes in general. In this paper we obtain optimal LRCs among these repeated-root constacyclic codes. Several infinite classes of optimal LRCs over a fixed alphabet are found. Under a further assumption that the ambient space of the repeated-root constacyclic codes is a chain ring, we show that there is no other optimal LRC.

[8]  arXiv:2111.12494 [pdf, other]
Title: Time-Energy-Constrained Closed-Loop FBL Communication for Dependable MEC
Comments: Accepted for publication at CSCN 2021
Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)

The deployment of multi-access edge computing (MEC) is paving the way towards pervasive intelligence in future 6G networks. This new paradigm also proposes emerging requirements of dependable communications, which goes beyond the ultra-reliable low latency communication (URLLC), focusing on the performance of a closed loop instead of that of an unidirectional link. This work studies the simple but efficient one-shot transmission scheme, investigating the closed-loop-reliability-optimal policy of blocklength allocation under stringent time and energy constraints.

[9]  arXiv:2111.12497 [pdf, ps, other]
Title: Performance of Reconfigurable Intelligent Surfaces in the Presence of Generalized Gaussian Noise
Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)

In this letter, we investigate the performance of reconfigurable intelligent surface (RIS)-assisted communications, under the assumption of generalized Gaussian noise (GGN), over Rayleigh fading channels. Specifically, we consider an RIS, equipped with $N$ reflecting elements, and derive a novel closed-form expression for the symbol error rate (SER) of arbitrary modulation schemes. The usefulness of the derived new expression is that it can be used to capture the SER performance in the presence of special additive noise distributions such as Gamma, Laplacian, and Gaussian noise. These special cases are also considered and their associated asymptotic SER expressions are derived, and then employed to quantify the achievable diversity order of the system. The theoretical framework is corroborated by numerical results, which reveal that the shaping parameter of the GGN ($\alpha$) has a negligible effect on the diversity order of RIS-assisted systems, particularly for large $\alpha$ values. Accordingly, the maximum achievable diversity order is determined by $N$.

[10]  arXiv:2111.12675 [pdf, other]
Title: The Surprising Benefits of Hysteresis in Unlimited Sampling: Theory, Algorithms and Experiments
Comments: 24 pages
Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)

The Unlimited Sensing Framework (USF) was recently introduced to overcome the sensor saturation bottleneck in conventional digital acquisition systems. At its core, the USF allows for high-dynamic-range (HDR) signal reconstruction by converting a continuous-time signal into folded, low-dynamic-range (LDR), modulo samples. HDR reconstruction is then carried out by algorithmic unfolding of the folded samples. In hardware, however, implementing an ideal modulo folding requires careful calibration, analog design and high precision. At the interface of theory and practice, this paper explores a computational sampling strategy that relaxes strict hardware requirements by compensating them via a novel, mathematically guaranteed recovery method. Our starting point is a generalized model for USF. The generalization relies on two new parameters modeling hysteresis and folding transients} in addition to the modulo threshold. Hysteresis accounts for the mismatch between the reset threshold and the amplitude displacement at the folding time and we refer to a continuous transition period in the implementation of a reset as folding transient. Both these effects are motivated by our hardware experiments and also occur in previous, domain-specific applications. We show that the effect of hysteresis is beneficial for the USF and we leverage it to derive the first recovery guarantees in the context of our generalized USF model. Additionally, we show how the proposed recovery can be directly generalized for the case of lower sampling rates. Our theoretical work is corroborated by hardware experiments that are based on a hysteresis enabled, modulo ADC testbed comprising off-the-shelf electronic components. Thus, by capitalizing on a collaboration between hardware and algorithms, our paper enables an end-to-end pipeline for HDR sampling allowing more flexible hardware implementations.

Cross-lists for Thu, 25 Nov 21

[11]  arXiv:2111.12312 (cross-list from math.PR) [pdf, ps, other]
Title: Lossy Compression of General Random Variables
Subjects: Probability (math.PR); Information Theory (cs.IT)

This paper is concerned with the lossy compression of general random variables, specifically with rate-distortion theory and quantization of random variables taking values in general measurable spaces such as, e.g., manifolds and fractal sets. Manifold structures are prevalent in data science, e.g., in compressed sensing, machine learning, image processing, and handwritten digit recognition. Fractal sets find application in image compression and in the modeling of Ethernet traffic. Our main contributions are bounds on the rate-distortion function and the quantization error. These bounds are very general and essentially only require the existence of reference measures satisfying certain regularity conditions in terms of small ball probabilities. To illustrate the wide applicability of our results, we particularize them to random variables taking values in i) manifolds, namely, hyperspheres and Grassmannians, and ii) self-similar sets characterized by iterated function systems satisfying the weak separation property.

[12]  arXiv:2111.12550 (cross-list from cs.HC) [pdf, other]
Title: A Worker-Task Specialization Model for Crowdsourcing: Efficient Inference and Fundamental Limits
Subjects: Human-Computer Interaction (cs.HC); Information Theory (cs.IT); Machine Learning (cs.LG); Machine Learning (stat.ML)

Crowdsourcing system has emerged as an effective platform to label data with relatively low cost by using non-expert workers. However, inferring correct labels from multiple noisy answers on data has been a challenging problem, since the quality of answers varies widely across tasks and workers. Many previous works have assumed a simple model where the order of workers in terms of their reliabilities is fixed across tasks, and focused on estimating the worker reliabilities to aggregate answers with different weights. We propose a highly general $d$-type worker-task specialization model in which the reliability of each worker can change depending on the type of a given task, where the number $d$ of types can scale in the number of tasks. In this model, we characterize the optimal sample complexity to correctly infer labels with any given recovery accuracy, and propose an inference algorithm achieving the order-wise optimal bound. We conduct experiments both on synthetic and real-world datasets, and show that our algorithm outperforms the existing algorithms developed based on strict model assumptions.

Replacements for Thu, 25 Nov 21

[13]  arXiv:1910.02534 (replaced) [pdf, other]
Title: The CEO problem with inter-block memory
Journal-ref: IEEE Transactions on Information Theory, v. 67, No. 12, pp. 7752--7768, Dec. 2021
Subjects: Information Theory (cs.IT); Systems and Control (eess.SY)
[14]  arXiv:2001.04194 (replaced) [pdf, other]
Title: Cascaded Coded Distributed Computing Schemes Based on Placement Delivery Arrays
Subjects: Information Theory (cs.IT)
[15]  arXiv:2003.04175 (replaced) [pdf, other]
Title: Phase Transition Analysis for Covariance Based Massive Random Access with Massive MIMO
Comments: Accepted in IEEE Transactions on Information Theory
Subjects: Information Theory (cs.IT)
[16]  arXiv:2012.00968 (replaced) [pdf, other]
Title: Reconfigurable Intelligent Surfaces in Action for Non-Terrestrial Networks
Comments: 7 pages, 6 figures
Subjects: Information Theory (cs.IT); Machine Learning (cs.LG); Signal Processing (eess.SP)
[17]  arXiv:2103.00782 (replaced) [pdf, other]
Title: Sparse Activity Detection in Multi-Cell Massive MIMO Exploiting Channel Large-Scale Fading
Comments: This is the final version published in IEEE Transactions on Signal Processing
Subjects: Information Theory (cs.IT)
[18]  arXiv:2107.10294 (replaced) [pdf, other]
Title: User-Centric Perspective in Random Access Cell-Free Aided by Spatial Separability
Comments: 14 pages, 8 figures
Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)
[19]  arXiv:2107.14669 (replaced) [pdf, ps, other]
Title: Representing preorders with injective monotones
Subjects: Information Theory (cs.IT)
[20]  arXiv:2108.13033 (replaced) [pdf, ps, other]
Title: Resource Allocation for Active IRS-Assisted Multiuser Communication Systems
Comments: 3 figures, submitted to Asilomar 2021
Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)
[21]  arXiv:1612.05924 (replaced) [src]
Title: Asymmetric Hat Game with three players and three colors
Authors: Theo van Uem
Comments: it is now part of arXiv:1612.00276 (Ebert's asymmetric Hat Game)
Subjects: Combinatorics (math.CO); Discrete Mathematics (cs.DM); Information Theory (cs.IT)
[22]  arXiv:1704.04244 (replaced) [src]
Title: General three person two color Hat Game
Authors: Theo van Uem
Comments: it is now part of arXiv:1612.00276 (Ebert's asymmetric Hat Game)
Subjects: Combinatorics (math.CO); Discrete Mathematics (cs.DM); Information Theory (cs.IT)
[23]  arXiv:2102.09159 (replaced) [pdf, other]
Title: Robust and Differentially Private Mean Estimation
Comments: 58 pages, 2 figures, both exponential time and efficient algorithms no longer require a known bound on the true mean
Subjects: Machine Learning (cs.LG); Cryptography and Security (cs.CR); Information Theory (cs.IT); Machine Learning (stat.ML)
[24]  arXiv:2106.03969 (replaced) [pdf, other]
Title: Chow-Liu++: Optimal Prediction-Centric Learning of Tree Ising Models
Comments: 49 pages, 3 figures, to appear in FOCS'21
Subjects: Machine Learning (cs.LG); Data Structures and Algorithms (cs.DS); Information Theory (cs.IT); Statistics Theory (math.ST)
[ total of 24 entries: 1-24 ]
[ showing up to 2000 entries per page: fewer | more ]

Disable MathJax (What is MathJax?)

Links to: arXiv, form interface, find, cs, recent, 2111, contact, help  (Access key information)