We gratefully acknowledge support from
the Simons Foundation and member institutions.

Information Theory

New submissions

[ total of 19 entries: 1-19 ]
[ showing up to 2000 entries per page: fewer | more ]

New submissions for Fri, 3 Feb 23

[1]  arXiv:2302.00819 [pdf, ps, other]
Title: Introduction to Arithmetic Coding -- Theory and Practice
Authors: Amir Said
Comments: Hewlett-Packard Laboratories Report; Chapter in Lossless Compression Handbook (ed. K. Sayood), Academic Press
Subjects: Information Theory (cs.IT); Multimedia (cs.MM)

This introduction to arithmetic coding is divided in two parts. The first explains how and why arithmetic coding works. We start presenting it in very general terms, so that its simplicity is not lost under layers of implementation details. Next, we show some of its basic properties, which are later used in the computational techniques required for a practical implementation. In the second part, we cover the practical implementation aspects, including arithmetic operations with low precision, the subdivision of coding and modeling, and the realization of adaptive encoders. We also analyze the arithmetic coding computational complexity, and techniques to reduce it.

[2]  arXiv:2302.00847 [pdf, ps, other]
Title: Low-Complexity Precoding for Extremely Large-Scale MIMO Over Non-Stationary Channels
Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)

Extremely large-scale multiple-input-multiple-output (XL-MIMO) is a promising technology for the future sixth-generation (6G) networks to achieve higher performance. In practice, various linear precoding schemes, such as zero-forcing (ZF) and regularized zero-forcing (RZF) precoding, are capable of achieving both large spectral efficiency (SE) and low bit error rate (BER) in traditional massive MIMO (mMIMO) systems. However, these methods are not efficient in extremely large-scale regimes due to the inherent spatial non-stationarity and high computational complexity. To address this problem, we investigate a low-complexity precoding algorithm, e.g., randomized Kaczmarz (rKA), taking into account the spatial non-stationary properties in XL-MIMO systems. Furthermore, we propose a novel mode of randomization, i.e., sampling without replacement rKA (SwoR-rKA), which enjoys a faster convergence speed than the rKA algorithm. Besides, the closed-form expression of SE considering the interference between subarrays in downlink XL-MIMO systems is derived. Numerical results show that the complexity given by both rKA and SwoR-rKA algorithms has 51.3% reduction than the traditional RZF algorithm with similar SE performance. More importantly, our algorithms can effectively reduce the BER when the transmitter has imperfect channel estimation.

[3]  arXiv:2302.00891 [pdf, other]
Title: Role of Bootstrap Averaging in Generalized Approximate Message Passing
Comments: 6 pages, 5 figures
Subjects: Information Theory (cs.IT); Disordered Systems and Neural Networks (cond-mat.dis-nn); Statistics Theory (math.ST)

Generalized approximate message passing (GAMP) is a computationally efficient algorithm for estimating an unknown signal \(w_0\in\mathbb{R}^N\) from a random linear measurement \(y= Xw_0 + \epsilon\in\mathbb{R}^M\), where \(X\in\mathbb{R}^{M\times N}\) is a known measurement matrix and \(\epsilon\) is the noise vector. The salient feature of GAMP is that it can provide an unbiased estimator \(\hat{r}^{\rm G}\sim\mathcal{N}(w_0, \hat{s}^2I_N)\), which can be used for various hypothesis-testing methods. In this study, we consider the bootstrap average of an unbiased estimator of GAMP for the elastic net. By numerically analyzing the state evolution of \emph{approximate message passing with resampling}, which has been proposed for computing bootstrap statistics of the elastic net estimator, we investigate when the bootstrap averaging reduces the variance of the unbiased estimator and the effect of optimizing the size of each bootstrap sample and hyperparameter of the elastic net regularization in the asymptotic setting \(M, N\to\infty, M/N\to\alpha\in(0,\infty)\). The results indicate that bootstrap averaging effectively reduces the variance of the unbiased estimator when the actual data generation process is inconsistent with the sparsity assumption of the regularization and the sample size is small. Furthermore, we find that when \(w_0\) is less sparse, and the data size is small, the system undergoes a phase transition. The phase transition indicates the existence of the region where the ensemble average of unbiased estimators of GAMP for the elastic net norm minimization problem yields the unbiased estimator with the minimum variance.

[4]  arXiv:2302.00906 [pdf, ps, other]
Title: New Constructions of Optimal Binary LCD Codes
Comments: 28 pages
Subjects: Information Theory (cs.IT)

Linear complementary dual (LCD) codes can provide an optimum linear coding solution for the two-user binary adder channel. LCD codes also can be used to against side-channel attacks and fault non-invasive attacks. Let $d_{LCD}(n, k)$ denote the maximum value of $d$ for which a binary $[n,k, d]$ LCD code exists. In \cite{BS21}, Bouyuklieva conjectured that $d_{LCD}(n+1, k)=d_{LCD}(n, k)$ or $d_{LCD}(n, k) + 1$ for any lenth $n$ and dimension $k \ge 2$. In this paper, we first prove Bouyuklieva's conjecture \cite{BS21} by constructing a binary $[n,k,d-1]$ LCD codes from a binary $[n+1,k,d]$ $LCD_{o,e}$ code, when $d \ge 3$ and $k \ge 2$. Then we provide a distance lower bound for binary LCD codes by expanded codes, and use this bound and some methods such as puncturing, shortening, expanding and extension, we construct some new binary LCD codes. Finally, we improve some previously known values of $d_{LCD}(n, k)$ of lengths $38 \le n \le 40$ and dimensions $9 \le k \le 15$. We also obtain some values of $d_{LCD}(n, k)$ with $41 \le n \le 50$ and $6 \le k \le n-6$.

[5]  arXiv:2302.00979 [pdf, ps, other]
Title: Maximum weight codewords of a linear rank metric code
Subjects: Information Theory (cs.IT); Combinatorics (math.CO)

Let $\mathcal{C}\subseteq \mathbb{F}_{q^m}^n$ be an $\mathbb{F}_{q^m}$-linear non-degenerate rank metric code with dimension $k$. In this paper we investigate the problem of determining the number $M(\mathcal{C})$ of codewords in $\mathcal{C}$ with maximum weight, that is $\min\{m,n\}$, and to characterize those with the maximum and the minimum values of $M(\mathcal{C})$.

[6]  arXiv:2302.01035 [pdf, other]
Title: Deep Learning Based Predictive Beamforming Design
Comments: Accepted in IEEE Transactions on Vehicular Technology
Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)

This paper investigates deep learning techniques to predict transmit beamforming based on only historical channel data without current channel information in the multiuser multiple-input-single-output downlink. This will significantly reduce the channel estimation overhead and improve the spectrum efficiency especially in high-mobility vehicular communications. Specifically, we propose a joint learning framework that incorporates channel prediction and power optimization, and produces prediction for transmit beamforming directly. In addition, we propose to use the attention mechanism in the Long Short-Term Memory Recurrent Neural Networks to improve the accuracy of channel prediction. Simulation results using both a simple autoregressive process model and the more realistic 3GPP spatial channel model verify that our proposed predictive beamforming scheme can significantly improve the effective spectrum efficiency compared to traditional channel estimation and the method that separately predicts channel and then optimizes beamforming.

[7]  arXiv:2302.01093 [pdf, other]
Title: Energy savings under performance constraints via carrier shutdown with Bayesian learning
Subjects: Information Theory (cs.IT)

By shutting down frequency carriers, the power consumed by a base station can be considerably reduced. However, this typically comes with traffic performance degradation, as the congestion on the remaining active carriers is increased. We leverage a hysteresis carrier shutdown policy that attempts to keep the average traffic load on each sector within a certain min/max threshold pair. We propose a closed-loop Bayesian method optimizing such thresholds on a sector basis and aiming at minimizing the power consumed by the power amplifiers while maintaining the probability that KPI's are acceptable above a certain value. We tested our approach in a live customer 4G network. The power consumption at the base station was reduced by 11% and the selected KPI's met the predefined targets.

[8]  arXiv:2302.01132 [pdf, other]
Title: Real-time Remote Reconstruction of a Markov Source and Actuation over Wireless
Subjects: Information Theory (cs.IT); Networking and Internet Architecture (cs.NI); Signal Processing (eess.SP); Systems and Control (eess.SY)

In this work, we study the problem of real-time tracking and reconstruction of an information source with the purpose of actuation. A device monitors an $N$-state Markov process and transmits status updates to a receiver over a wireless erasure channel. We consider a set of joint sampling and transmission policies, including a semantics-aware one, and we study their performance with respect to relevant metrics. Specifically, we investigate the real-time reconstruction error and its variance, the consecutive error, the cost of memory error, and the cost of actuation error. Furthermore, we propose a randomized stationary sampling and transmission policy and derive closed-form expressions for all aforementioned metrics. We then formulate an optimization problem for minimizing the real-time reconstruction error subject to a sampling cost constraint. Our results show that in the scenario of constrained sampling generation, the optimal randomized stationary policy outperforms all other sampling policies when the source is rapidly evolving. Otherwise, the semantics-aware policy performs the best.

[9]  arXiv:2302.01153 [pdf, other]
Title: Near-Field Integrated Sensing and Communications
Comments: 14 pages, 4 figures
Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)

A near-field integrated sensing and communications (NF-ISAC) framework is proposed, where both a sensing target and multiple communication users are located within the near-field region. An extended multiple signal classification (MUSIC) algorithm is proposed for the joint distance and angle estimation. The performance tradeoff between communication and sensing is characterized by the proposed two-stage algorithm. Numerical results reveal 1) the additional advantage of the NF-ISAC in terms of distance estimation compared to the conventional far-field ISAC, and 2) the tradeoff between the distance and angle estimation.

[10]  arXiv:2302.01314 [pdf, ps, other]
Title: Universal Secure Source Encryption under Side-Channel Attacks
Comments: 11 pages, 3 figures. arXiv admin note: substantial text overlap with arXiv:2201.11670, arXiv:1801.02563
Subjects: Information Theory (cs.IT)

We study the universal coding under side-channel attacks posed and investigated by Santoso and Oohama (2021). They proposed a theoretical security model for Shannon cipher system under side-channel attacks, where the adversary is not only allowed to collect ciphertexts by eavesdropping the public communication channel, but is also allowed to collect the physical information leaked by the devices where the cipher system is implemented on such as running time, power consumption, electromagnetic radiation, etc. For any distributions of the plain text, any noisy channels through which the adversary observe the corrupted version of the key, and any measurement device used for collecting the physical information, we can derive an achievable rate region for reliability and security such that if we compress the ciphertext using an affine encoder with rate within the achievable rate region, then: (1) anyone with secret key will be able to decrypt and decode the ciphertext correctly, but (2) any adversary who obtains the ciphertext and also the side physical information will not be able to obtain any information about the hidden source as long as the leaked physical information is encoded with a rate within the rate region.

Cross-lists for Fri, 3 Feb 23

[11]  arXiv:2302.00919 (cross-list from eess.SP) [pdf, other]
Title: QCM-SGM+: Improved Quantized Compressed Sensing With Score-Based Generative Models for General Sensing Matrices
Comments: arXiv admin note: substantial text overlap with arXiv:2211.13006
Subjects: Signal Processing (eess.SP); Information Theory (cs.IT); Machine Learning (cs.LG)

In realistic compressed sensing (CS) scenarios, the obtained measurements usually have to be quantized to a finite number of bits before transmission and/or storage, thus posing a challenge in recovery, especially for extremely coarse quantization such as 1-bit sign measurements. Recently Meng & Kabashima proposed an efficient quantized compressed sensing algorithm called QCS-SGM using the score-based generative models as an implicit prior. Thanks to the power of score-based generative models in capturing the rich structure of the prior, QCS-SGM achieves remarkably better performances than previous quantized CS methods. However, QCS-SGM is restricted to (approximately) row-orthogonal sensing matrices since otherwise the likelihood score becomes intractable. To address this challenging problem, in this paper we propose an improved version of QCS-SGM, which we call QCS-SGM+, which also works well for general matrices. The key idea is a Bayesian inference perspective of the likelihood score computation, whereby an expectation propagation algorithm is proposed to approximately compute the likelihood score. Experiments on a variety of baseline datasets demonstrate that the proposed QCS-SGM+ outperforms QCS-SGM by a large margin when sensing matrices are far from row-orthogonal.

[12]  arXiv:2302.00920 (cross-list from math.NT) [pdf, ps, other]
Title: Certain Diagonal Equations and Conflict-Avoiding Codes of Prime Lengths
Subjects: Number Theory (math.NT); Information Theory (cs.IT)

We study the construction of optimal conflict-avoiding codes (CAC) from a number theoretical point of view. The determination of the size of optimal CAC of prime length $p$ and weight 3 is formulated in terms of the solvability of certain twisted Fermat equations of the form $g^2 X^{\ell} + g Y^{\ell} + 1 = 0$ over the finite field $\mathbb{F}_{p}$ for some primitive root $g$ modulo $p.$ We treat the problem of solving the twisted Fermat equations in a more general situation by allowing the base field to be any finite extension field $\mathbb{F}_q$ of $\mathbb{F}_{p}.$ We show that for $q$ greater than a lower bound of the order of magnitude $O(\ell^2)$ there exists a generator $g$ of $\mathbb{F}_{q}^{\times}$ such that the equation in question is solvable over $\mathbb{F}_{q}.$ Using our results we are able to contribute new results to the construction of optimal CAC of prime lengths and weight $3.$

[13]  arXiv:2302.01333 (cross-list from cs.LG) [pdf, other]
Title: Lower Bounds for Learning in Revealing POMDPs
Subjects: Machine Learning (cs.LG); Information Theory (cs.IT); Statistics Theory (math.ST); Machine Learning (stat.ML)

This paper studies the fundamental limits of reinforcement learning (RL) in the challenging \emph{partially observable} setting. While it is well-established that learning in Partially Observable Markov Decision Processes (POMDPs) requires exponentially many samples in the worst case, a surge of recent work shows that polynomial sample complexities are achievable under the \emph{revealing condition} -- A natural condition that requires the observables to reveal some information about the unobserved latent states. However, the fundamental limits for learning in revealing POMDPs are much less understood, with existing lower bounds being rather preliminary and having substantial gaps from the current best upper bounds.
We establish strong PAC and regret lower bounds for learning in revealing POMDPs. Our lower bounds scale polynomially in all relevant problem parameters in a multiplicative fashion, and achieve significantly smaller gaps against the current best upper bounds, providing a solid starting point for future studies. In particular, for \emph{multi-step} revealing POMDPs, we show that (1) the latent state-space dependence is at least $\Omega(S^{1.5})$ in the PAC sample complexity, which is notably harder than the $\widetilde{\Theta}(S)$ scaling for fully-observable MDPs; (2) Any polynomial sublinear regret is at least $\Omega(T^{2/3})$, suggesting its fundamental difference from the \emph{single-step} case where $\widetilde{O}(\sqrt{T})$ regret is achievable. Technically, our hard instance construction adapts techniques in \emph{distribution testing}, which is new to the RL literature and may be of independent interest.

Replacements for Fri, 3 Feb 23

[14]  arXiv:2204.01612 (replaced) [pdf, other]
Title: Neural Estimation of the Rate-Distortion Function With Applications to Operational Source Coding
Subjects: Information Theory (cs.IT); Machine Learning (cs.LG); Machine Learning (stat.ML)
[15]  arXiv:2206.05407 (replaced) [pdf, other]
Title: Opportunistic Routing aided Cooperative Communication MRC Network with Energy-Harvesting Nodes
Comments: arXiv admin note: text overlap with arXiv:2205.06482
Subjects: Information Theory (cs.IT)
[16]  arXiv:2210.11747 (replaced) [pdf, other]
Title: A Finite Blocklength Approach for Wireless Hierarchical Federated Learning in the Presence of Physical Layer Security
Comments: submitted to IEEE INFOCOM Wireless-Sec 2023
Subjects: Information Theory (cs.IT)
[17]  arXiv:2107.11914 (replaced) [pdf, other]
Title: Stabilizer codes for Open Quantum Systems
Subjects: Quantum Physics (quant-ph); Information Theory (cs.IT)
[18]  arXiv:2204.12723 (replaced) [pdf, ps, other]
Title: Information-theoretic limitations of data-based price discrimination
Comments: In the new version, we have (1) added a simulation and empirical study and (2) fixed some minor issues and improved the clarity
Subjects: Computer Science and Game Theory (cs.GT); Information Theory (cs.IT); Machine Learning (cs.LG); Econometrics (econ.EM); Theoretical Economics (econ.TH)
[19]  arXiv:2207.03619 (replaced) [pdf, ps, other]
Title: Constructions and restrictions for balanced splittable Hadamard matrices
Comments: Minor corrections
Subjects: Combinatorics (math.CO); Information Theory (cs.IT)
[ total of 19 entries: 1-19 ]
[ showing up to 2000 entries per page: fewer | more ]

Disable MathJax (What is MathJax?)

Links to: arXiv, form interface, find, cs, recent, 2302, contact, help  (Access key information)