Information Theory
New submissions
[ showing up to 2000 entries per page: fewer  more ]
New submissions for Fri, 3 Feb 23
 [1] arXiv:2302.00819 [pdf, ps, other]

Title: Introduction to Arithmetic Coding  Theory and PracticeAuthors: Amir SaidComments: HewlettPackard Laboratories Report; Chapter in Lossless Compression Handbook (ed. K. Sayood), Academic PressSubjects: Information Theory (cs.IT); Multimedia (cs.MM)
This introduction to arithmetic coding is divided in two parts. The first explains how and why arithmetic coding works. We start presenting it in very general terms, so that its simplicity is not lost under layers of implementation details. Next, we show some of its basic properties, which are later used in the computational techniques required for a practical implementation. In the second part, we cover the practical implementation aspects, including arithmetic operations with low precision, the subdivision of coding and modeling, and the realization of adaptive encoders. We also analyze the arithmetic coding computational complexity, and techniques to reduce it.
 [2] arXiv:2302.00847 [pdf, ps, other]

Title: LowComplexity Precoding for Extremely LargeScale MIMO Over NonStationary ChannelsSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
Extremely largescale multipleinputmultipleoutput (XLMIMO) is a promising technology for the future sixthgeneration (6G) networks to achieve higher performance. In practice, various linear precoding schemes, such as zeroforcing (ZF) and regularized zeroforcing (RZF) precoding, are capable of achieving both large spectral efficiency (SE) and low bit error rate (BER) in traditional massive MIMO (mMIMO) systems. However, these methods are not efficient in extremely largescale regimes due to the inherent spatial nonstationarity and high computational complexity. To address this problem, we investigate a lowcomplexity precoding algorithm, e.g., randomized Kaczmarz (rKA), taking into account the spatial nonstationary properties in XLMIMO systems. Furthermore, we propose a novel mode of randomization, i.e., sampling without replacement rKA (SwoRrKA), which enjoys a faster convergence speed than the rKA algorithm. Besides, the closedform expression of SE considering the interference between subarrays in downlink XLMIMO systems is derived. Numerical results show that the complexity given by both rKA and SwoRrKA algorithms has 51.3% reduction than the traditional RZF algorithm with similar SE performance. More importantly, our algorithms can effectively reduce the BER when the transmitter has imperfect channel estimation.
 [3] arXiv:2302.00891 [pdf, other]

Title: Role of Bootstrap Averaging in Generalized Approximate Message PassingAuthors: Takashi TakahashiComments: 6 pages, 5 figuresSubjects: Information Theory (cs.IT); Disordered Systems and Neural Networks (condmat.disnn); Statistics Theory (math.ST)
Generalized approximate message passing (GAMP) is a computationally efficient algorithm for estimating an unknown signal \(w_0\in\mathbb{R}^N\) from a random linear measurement \(y= Xw_0 + \epsilon\in\mathbb{R}^M\), where \(X\in\mathbb{R}^{M\times N}\) is a known measurement matrix and \(\epsilon\) is the noise vector. The salient feature of GAMP is that it can provide an unbiased estimator \(\hat{r}^{\rm G}\sim\mathcal{N}(w_0, \hat{s}^2I_N)\), which can be used for various hypothesistesting methods. In this study, we consider the bootstrap average of an unbiased estimator of GAMP for the elastic net. By numerically analyzing the state evolution of \emph{approximate message passing with resampling}, which has been proposed for computing bootstrap statistics of the elastic net estimator, we investigate when the bootstrap averaging reduces the variance of the unbiased estimator and the effect of optimizing the size of each bootstrap sample and hyperparameter of the elastic net regularization in the asymptotic setting \(M, N\to\infty, M/N\to\alpha\in(0,\infty)\). The results indicate that bootstrap averaging effectively reduces the variance of the unbiased estimator when the actual data generation process is inconsistent with the sparsity assumption of the regularization and the sample size is small. Furthermore, we find that when \(w_0\) is less sparse, and the data size is small, the system undergoes a phase transition. The phase transition indicates the existence of the region where the ensemble average of unbiased estimators of GAMP for the elastic net norm minimization problem yields the unbiased estimator with the minimum variance.
 [4] arXiv:2302.00906 [pdf, ps, other]

Title: New Constructions of Optimal Binary LCD CodesComments: 28 pagesSubjects: Information Theory (cs.IT)
Linear complementary dual (LCD) codes can provide an optimum linear coding solution for the twouser binary adder channel. LCD codes also can be used to against sidechannel attacks and fault noninvasive attacks. Let $d_{LCD}(n, k)$ denote the maximum value of $d$ for which a binary $[n,k, d]$ LCD code exists. In \cite{BS21}, Bouyuklieva conjectured that $d_{LCD}(n+1, k)=d_{LCD}(n, k)$ or $d_{LCD}(n, k) + 1$ for any lenth $n$ and dimension $k \ge 2$. In this paper, we first prove Bouyuklieva's conjecture \cite{BS21} by constructing a binary $[n,k,d1]$ LCD codes from a binary $[n+1,k,d]$ $LCD_{o,e}$ code, when $d \ge 3$ and $k \ge 2$. Then we provide a distance lower bound for binary LCD codes by expanded codes, and use this bound and some methods such as puncturing, shortening, expanding and extension, we construct some new binary LCD codes. Finally, we improve some previously known values of $d_{LCD}(n, k)$ of lengths $38 \le n \le 40$ and dimensions $9 \le k \le 15$. We also obtain some values of $d_{LCD}(n, k)$ with $41 \le n \le 50$ and $6 \le k \le n6$.
 [5] arXiv:2302.00979 [pdf, ps, other]

Title: Maximum weight codewords of a linear rank metric codeSubjects: Information Theory (cs.IT); Combinatorics (math.CO)
Let $\mathcal{C}\subseteq \mathbb{F}_{q^m}^n$ be an $\mathbb{F}_{q^m}$linear nondegenerate rank metric code with dimension $k$. In this paper we investigate the problem of determining the number $M(\mathcal{C})$ of codewords in $\mathcal{C}$ with maximum weight, that is $\min\{m,n\}$, and to characterize those with the maximum and the minimum values of $M(\mathcal{C})$.
 [6] arXiv:2302.01035 [pdf, other]

Title: Deep Learning Based Predictive Beamforming DesignComments: Accepted in IEEE Transactions on Vehicular TechnologySubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
This paper investigates deep learning techniques to predict transmit beamforming based on only historical channel data without current channel information in the multiuser multipleinputsingleoutput downlink. This will significantly reduce the channel estimation overhead and improve the spectrum efficiency especially in highmobility vehicular communications. Specifically, we propose a joint learning framework that incorporates channel prediction and power optimization, and produces prediction for transmit beamforming directly. In addition, we propose to use the attention mechanism in the Long ShortTerm Memory Recurrent Neural Networks to improve the accuracy of channel prediction. Simulation results using both a simple autoregressive process model and the more realistic 3GPP spatial channel model verify that our proposed predictive beamforming scheme can significantly improve the effective spectrum efficiency compared to traditional channel estimation and the method that separately predicts channel and then optimizes beamforming.
 [7] arXiv:2302.01093 [pdf, other]

Title: Energy savings under performance constraints via carrier shutdown with Bayesian learningAuthors: Lorenzo Maggi, Claudiu Mihailescu, Qike Cao, Simo Aaltonen, Ryo Koblitz, Maunu Holma, Samuele Macchi, Maria Elena Ruggieri, Igor Korenev, Bjarne KlausenSubjects: Information Theory (cs.IT)
By shutting down frequency carriers, the power consumed by a base station can be considerably reduced. However, this typically comes with traffic performance degradation, as the congestion on the remaining active carriers is increased. We leverage a hysteresis carrier shutdown policy that attempts to keep the average traffic load on each sector within a certain min/max threshold pair. We propose a closedloop Bayesian method optimizing such thresholds on a sector basis and aiming at minimizing the power consumed by the power amplifiers while maintaining the probability that KPI's are acceptable above a certain value. We tested our approach in a live customer 4G network. The power consumption at the base station was reduced by 11% and the selected KPI's met the predefined targets.
 [8] arXiv:2302.01132 [pdf, other]

Title: Realtime Remote Reconstruction of a Markov Source and Actuation over WirelessSubjects: Information Theory (cs.IT); Networking and Internet Architecture (cs.NI); Signal Processing (eess.SP); Systems and Control (eess.SY)
In this work, we study the problem of realtime tracking and reconstruction of an information source with the purpose of actuation. A device monitors an $N$state Markov process and transmits status updates to a receiver over a wireless erasure channel. We consider a set of joint sampling and transmission policies, including a semanticsaware one, and we study their performance with respect to relevant metrics. Specifically, we investigate the realtime reconstruction error and its variance, the consecutive error, the cost of memory error, and the cost of actuation error. Furthermore, we propose a randomized stationary sampling and transmission policy and derive closedform expressions for all aforementioned metrics. We then formulate an optimization problem for minimizing the realtime reconstruction error subject to a sampling cost constraint. Our results show that in the scenario of constrained sampling generation, the optimal randomized stationary policy outperforms all other sampling policies when the source is rapidly evolving. Otherwise, the semanticsaware policy performs the best.
 [9] arXiv:2302.01153 [pdf, other]

Title: NearField Integrated Sensing and CommunicationsComments: 14 pages, 4 figuresSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
A nearfield integrated sensing and communications (NFISAC) framework is proposed, where both a sensing target and multiple communication users are located within the nearfield region. An extended multiple signal classification (MUSIC) algorithm is proposed for the joint distance and angle estimation. The performance tradeoff between communication and sensing is characterized by the proposed twostage algorithm. Numerical results reveal 1) the additional advantage of the NFISAC in terms of distance estimation compared to the conventional farfield ISAC, and 2) the tradeoff between the distance and angle estimation.
 [10] arXiv:2302.01314 [pdf, ps, other]

Title: Universal Secure Source Encryption under SideChannel AttacksComments: 11 pages, 3 figures. arXiv admin note: substantial text overlap with arXiv:2201.11670, arXiv:1801.02563Subjects: Information Theory (cs.IT)
We study the universal coding under sidechannel attacks posed and investigated by Santoso and Oohama (2021). They proposed a theoretical security model for Shannon cipher system under sidechannel attacks, where the adversary is not only allowed to collect ciphertexts by eavesdropping the public communication channel, but is also allowed to collect the physical information leaked by the devices where the cipher system is implemented on such as running time, power consumption, electromagnetic radiation, etc. For any distributions of the plain text, any noisy channels through which the adversary observe the corrupted version of the key, and any measurement device used for collecting the physical information, we can derive an achievable rate region for reliability and security such that if we compress the ciphertext using an affine encoder with rate within the achievable rate region, then: (1) anyone with secret key will be able to decrypt and decode the ciphertext correctly, but (2) any adversary who obtains the ciphertext and also the side physical information will not be able to obtain any information about the hidden source as long as the leaked physical information is encoded with a rate within the rate region.
Crosslists for Fri, 3 Feb 23
 [11] arXiv:2302.00919 (crosslist from eess.SP) [pdf, other]

Title: QCMSGM+: Improved Quantized Compressed Sensing With ScoreBased Generative Models for General Sensing MatricesComments: arXiv admin note: substantial text overlap with arXiv:2211.13006Subjects: Signal Processing (eess.SP); Information Theory (cs.IT); Machine Learning (cs.LG)
In realistic compressed sensing (CS) scenarios, the obtained measurements usually have to be quantized to a finite number of bits before transmission and/or storage, thus posing a challenge in recovery, especially for extremely coarse quantization such as 1bit sign measurements. Recently Meng & Kabashima proposed an efficient quantized compressed sensing algorithm called QCSSGM using the scorebased generative models as an implicit prior. Thanks to the power of scorebased generative models in capturing the rich structure of the prior, QCSSGM achieves remarkably better performances than previous quantized CS methods. However, QCSSGM is restricted to (approximately) roworthogonal sensing matrices since otherwise the likelihood score becomes intractable. To address this challenging problem, in this paper we propose an improved version of QCSSGM, which we call QCSSGM+, which also works well for general matrices. The key idea is a Bayesian inference perspective of the likelihood score computation, whereby an expectation propagation algorithm is proposed to approximately compute the likelihood score. Experiments on a variety of baseline datasets demonstrate that the proposed QCSSGM+ outperforms QCSSGM by a large margin when sensing matrices are far from roworthogonal.
 [12] arXiv:2302.00920 (crosslist from math.NT) [pdf, ps, other]

Title: Certain Diagonal Equations and ConflictAvoiding Codes of Prime LengthsSubjects: Number Theory (math.NT); Information Theory (cs.IT)
We study the construction of optimal conflictavoiding codes (CAC) from a number theoretical point of view. The determination of the size of optimal CAC of prime length $p$ and weight 3 is formulated in terms of the solvability of certain twisted Fermat equations of the form $g^2 X^{\ell} + g Y^{\ell} + 1 = 0$ over the finite field $\mathbb{F}_{p}$ for some primitive root $g$ modulo $p.$ We treat the problem of solving the twisted Fermat equations in a more general situation by allowing the base field to be any finite extension field $\mathbb{F}_q$ of $\mathbb{F}_{p}.$ We show that for $q$ greater than a lower bound of the order of magnitude $O(\ell^2)$ there exists a generator $g$ of $\mathbb{F}_{q}^{\times}$ such that the equation in question is solvable over $\mathbb{F}_{q}.$ Using our results we are able to contribute new results to the construction of optimal CAC of prime lengths and weight $3.$
 [13] arXiv:2302.01333 (crosslist from cs.LG) [pdf, other]

Title: Lower Bounds for Learning in Revealing POMDPsSubjects: Machine Learning (cs.LG); Information Theory (cs.IT); Statistics Theory (math.ST); Machine Learning (stat.ML)
This paper studies the fundamental limits of reinforcement learning (RL) in the challenging \emph{partially observable} setting. While it is wellestablished that learning in Partially Observable Markov Decision Processes (POMDPs) requires exponentially many samples in the worst case, a surge of recent work shows that polynomial sample complexities are achievable under the \emph{revealing condition}  A natural condition that requires the observables to reveal some information about the unobserved latent states. However, the fundamental limits for learning in revealing POMDPs are much less understood, with existing lower bounds being rather preliminary and having substantial gaps from the current best upper bounds.
We establish strong PAC and regret lower bounds for learning in revealing POMDPs. Our lower bounds scale polynomially in all relevant problem parameters in a multiplicative fashion, and achieve significantly smaller gaps against the current best upper bounds, providing a solid starting point for future studies. In particular, for \emph{multistep} revealing POMDPs, we show that (1) the latent statespace dependence is at least $\Omega(S^{1.5})$ in the PAC sample complexity, which is notably harder than the $\widetilde{\Theta}(S)$ scaling for fullyobservable MDPs; (2) Any polynomial sublinear regret is at least $\Omega(T^{2/3})$, suggesting its fundamental difference from the \emph{singlestep} case where $\widetilde{O}(\sqrt{T})$ regret is achievable. Technically, our hard instance construction adapts techniques in \emph{distribution testing}, which is new to the RL literature and may be of independent interest.
Replacements for Fri, 3 Feb 23
 [14] arXiv:2204.01612 (replaced) [pdf, other]

Title: Neural Estimation of the RateDistortion Function With Applications to Operational Source CodingSubjects: Information Theory (cs.IT); Machine Learning (cs.LG); Machine Learning (stat.ML)
 [15] arXiv:2206.05407 (replaced) [pdf, other]

Title: Opportunistic Routing aided Cooperative Communication MRC Network with EnergyHarvesting NodesComments: arXiv admin note: text overlap with arXiv:2205.06482Subjects: Information Theory (cs.IT)
 [16] arXiv:2210.11747 (replaced) [pdf, other]

Title: A Finite Blocklength Approach for Wireless Hierarchical Federated Learning in the Presence of Physical Layer SecurityComments: submitted to IEEE INFOCOM WirelessSec 2023Subjects: Information Theory (cs.IT)
 [17] arXiv:2107.11914 (replaced) [pdf, other]

Title: Stabilizer codes for Open Quantum SystemsSubjects: Quantum Physics (quantph); Information Theory (cs.IT)
 [18] arXiv:2204.12723 (replaced) [pdf, ps, other]

Title: Informationtheoretic limitations of databased price discriminationComments: In the new version, we have (1) added a simulation and empirical study and (2) fixed some minor issues and improved the claritySubjects: Computer Science and Game Theory (cs.GT); Information Theory (cs.IT); Machine Learning (cs.LG); Econometrics (econ.EM); Theoretical Economics (econ.TH)
 [19] arXiv:2207.03619 (replaced) [pdf, ps, other]

Title: Constructions and restrictions for balanced splittable Hadamard matricesComments: Minor correctionsSubjects: Combinatorics (math.CO); Information Theory (cs.IT)
[ showing up to 2000 entries per page: fewer  more ]
Disable MathJax (What is MathJax?)
Links to: arXiv, form interface, find, cs, recent, 2302, contact, help (Access key information)