We gratefully acknowledge support from
the Simons Foundation and member institutions.

Information Theory

New submissions

[ total of 26 entries: 1-26 ]
[ showing up to 2000 entries per page: fewer | more ]

New submissions for Wed, 8 Feb 23

[1]  arXiv:2302.03081 [pdf, ps, other]
Title: Permutation resemblance
Comments: 12 pages
Subjects: Information Theory (cs.IT); Combinatorics (math.CO); Number Theory (math.NT)

Motivated by the problem of constructing bijective maps with low differential uniformity, we introduce the notion of permutation resemblance of a function, which looks to measure the distance a given map is from being a permutation. We prove several results concerning permutation resemblance and show how it can be used to produce low differentially uniform bijections. We also study the permutation resemblance of planar functions, which over fields of odd characteristic are known not to be bijections and to have the optimal differential uniformity.

[2]  arXiv:2302.03188 [pdf, ps, other]
Title: Stacked Intelligent Metasurfaces Enabled Multiuser Beamforming in the Wave Domain
Comments: 6 pages, 4 figures, accepted by IEEE ICC 2023
Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)

Reconfigurable intelligent surface has recently emerged as a promising technology for reshaping the wireless environment by leveraging massive low-cost passive elements. Prior works mainly focus on a single-layer metasurface that lacks the capability of suppressing inter-user interference. By contrast, we propose in this paper a stacked intelligent metasurfaces (SIM)-enabled transceiver design for multiuser multiple-input single-output downlink communications. Specifically, a SIM having a multilayer structure is deployed at the base station to perform the transmit beamforming directly in the electromagnetic wave domain. As a result, the conventional digital beamforming and high-resolution analog-to-digital converters as well as the excessive number of radio-frequency chains are fully removed, which sharply reduces the hardware cost and energy consumption, while substantially decreasing the precoding delay benefiting from the computation at the speed of light. To this end, we formulate an optimization problem for maximizing the sum rate of all users by jointly designing the transmit power allocated to different users and the wave-based beamforming. Finally, numerical results based on a customized alternating optimization algorithm corroborate the effectiveness of our SIM-enabled wave-based beamforming design as compared to various benchmark schemes. Most notably, the wave-based beamforming is capable of decreasing the precoding delay by eight orders of magnitude compared to its digital counterpart.

[3]  arXiv:2302.03306 [pdf, other]
Title: Mismatched estimation of non-symmetric rank-one matrices corrupted by structured noise
Subjects: Information Theory (cs.IT); Machine Learning (cs.LG); Statistics Theory (math.ST)

We study the performance of a Bayesian statistician who estimates a rank-one signal corrupted by non-symmetric rotationally invariant noise with a generic distribution of singular values. As the signal-to-noise ratio and the noise structure are unknown, a Gaussian setup is incorrectly assumed. We derive the exact analytic expression for the error of the mismatched Bayes estimator and also provide the analysis of an approximate message passing (AMP) algorithm. The first result exploits the asymptotic behavior of spherical integrals for rectangular matrices and of low-rank matrix perturbations; the second one relies on the design and analysis of an auxiliary AMP. The numerical experiments show that there is a performance gap between the AMP and Bayes estimators, which is due to the incorrect estimation of the signal norm.

[4]  arXiv:2302.03335 [pdf, ps, other]
Title: Low-Latency Communication using Delay-Aware Relays Against Reactive Adversaries
Comments: 30 pages
Subjects: Information Theory (cs.IT)

This work addresses a reactive jamming attack on the low-latency messages of a victim, wherein the jammer deploys countermeasure detection mechanisms to change its strategy. We highlight that the existing schemes against reactive jammers use relays with instantaneous full-duplex (FD) radios to evade the attack. However, due to the limitation of the radio architecture of the FD helper, instantaneous forwarding may not be possible in practice, thereby leading to increased decoding complexity at the destination and a high detection probability at the adversary. Pointing at this drawback, we propose a delay-aware cooperative framework wherein the victim seeks assistance from a delay-aware FD helper to forward its messages to the destination within the latency constraints. In particular, we first model the processing delay at the helper based on its hardware architecture, and then propose two low-complexity mitigation schemes, wherein the victim and the helper share their uplink frequencies using appropriate energy-splitting factors. For both the schemes, we solve the optimization problems of computing the near-optimal energy-splitting factors that minimize the joint error rates at the destination. Finally, through analytical and simulation results, we show that the proposed schemes facilitate the victim in evading the jamming attack whilst deceiving the reactive adversary.

[5]  arXiv:2302.03452 [pdf, other]
Title: Cache-Aided Communication Schemes via Combinatorial Designs and their $q$-analogs
Comments: arXiv admin note: substantial text overlap with arXiv:2001.05438, arXiv:1901.06383
Subjects: Information Theory (cs.IT)

We consider the standard broadcast setup with a single server broadcasting information to a number of clients, each of which contains local storage (called cache) of some size, which can store some parts of the available files at the server. The centralized coded caching framework, consists of a caching phase and a delivery phase, both of which are carefully designed in order to use the cache and the channel together optimally. In prior literature, various combinatorial structures have been used to construct coded caching schemes. One of the chief drawbacks of many of these existing constructions is the large subpacketization level, which denotes the number of times a file should be split for the schemes to provide coding gain. In this work, using a new binary matrix model, we present several novel constructions for coded caching based on the various types of combinatorial designs and their $q$-analogs, which are also called subspace designs. While most of the schemes constructed in this work (based on existing designs) have a high cache requirement, they provide a rate that is either constant or decreasing, and moreover require competitively small levels of subpacketization, which is an extremely important feature in practical applications of coded caching. We also apply our constructions to the distributed computing framework of MapReduce, which consists of three phases, the Map phase, the Shuffle phase and the Reduce phase. Using our binary matrix framework, we present a new simple generic coded data shuffling scheme. Employing our designs-based constructions in conjunction with this new shuffling scheme, we obtain new coded computing schemes which have low file complexity, with marginally higher communication load compared to the optimal scheme for equivalent parameters. We show that our schemes can neatly extend to the scenario with full and partial stragglers also.

[6]  arXiv:2302.03524 [pdf, other]
Title: Multiple Key-cast over Networks
Subjects: Information Theory (cs.IT)

The multicast key-dissemination problem over noiseless networks, introduced by Langberg and Effros [ITW 2022], here called the ``key-cast'' problem, captures the task of disseminating a shared secret random key to a set of terminals over a given network. Unlike traditional communication, where messages must be delivered from source to destination(s) unchanged, key-cast is more flexible since key-cast need not require source reconstruction at destination nodes. For example, the distributed keys can be mixtures of sources from which the sources themselves may be unrecoverable.
The work at hand considers key dissemination in the single-source, multiple-multicast network coding setting, i.e., the ``multiple key-cast'' problem. Here, distinct keys are to be simultaneously transmitted from a single source node to multiple terminal sets, one shared random key per multicast set. Scenarios include the secure setting, in which only the source and intended destinations gain information about a given key; and the non-secure setting in which the only requirement is that the knowledge of one key does not reveal information about another. In both settings, we present combinatorial conditions for key dissemination and design corresponding multiple key-cast schemes. In addition, we compare the multiple key-cast rate with and without the restriction of source reconstruction, the former corresponding to traditional forms of communication; key-cast achieves a strict advantage in rate when source reconstruction is relaxed.

[7]  arXiv:2302.03549 [pdf, other]
Title: An Information Bottleneck Approach to Secrecy-Preserving Gaussian Mixture Classification
Subjects: Information Theory (cs.IT)

The information bottleneck (IB) approach, initially introduced by [1] to assess the compression-relevance tradeoff for a remote source coding problem in communication, quickly gains popularity recently in its application to modern machine learning (ML). Unlike the use of most - if not all - IB in the literature, either for the analysis of, say deep neural networks, or as an optimization objective, in this paper, we propose to address the secrecy issue in ML, by considering the fundamental model of Gaussian mixture classification. We derive, for the first time, closed-form achievable bounds for the IB problem under the above setting, and provide precise characterization of the underlying performance-secrecy tradeoff. Experiments on both synthetic and real-world data are performed to confirm the satisfactory performance of the proposed scheme.

[8]  arXiv:2302.03574 [pdf, other]
Title: A Dominant Interferer plus Mean Field-based Approximation for SINR Meta Distribution in Wireless Networks
Subjects: Information Theory (cs.IT); Systems and Control (eess.SY)

This paper proposes a novel approach for computing the meta distribution of the signal-to-interference-plus-noise ratio (SINR) for the downlink transmission in a wireless network with Rayleigh fading. The novel approach relies on an approximation mix of exact and mean-field analysis of interference (dominant interferer-based approximation) to reduce the complexity of analysis and enhance tractability. In particular, the proposed approximation omits the need to compute the first or the second moment of the SINR that is used in the beta approximation typically adopted in the literature but requires of computing the joint distance distributions. We first derive the proposed approximation based on a Poisson point process (PPP) network with a standard path-loss and Rayleigh fading and then illustrate its accuracy and operability in another four widely used point processes: Poisson bipolar network, Mat\'{e}rn cluster process (MCP), $K$-tier PPP and Poisson line Cox process (PLCP). Specifically, we obtain the SINR meta distribution for PLCP networks for the first time. Even though the proposed approximation looks simple but it shows good matching in comparison to the popular beta approximation as well as the Monte-Carlo simulations, which opens the door to adopting this approximation in more advanced network architectures.

[9]  arXiv:2302.03656 [pdf, ps, other]
Title: Revealing the Impact of SIC in NOMA-ISAC
Comments: 5 pages
Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)

The impact of successive interference cancellation (SIC) in non-orthogonal multiple access integrated sensing and communications (NOMA-ISAC) is analyzed. A two-stage SIC-based framework is proposed to deal with the inter-communication user and inter-functionality interferences. The performance of sensing and communications (S\&C) is analyzed for two SIC orders, i.e., the communications-centric SIC and the sensing-centric SIC. For each design, diversity orders, high signal-to-noise ratio (SNR) slopes, and high-SNR power offsets of the sensing rate (SR) and communication rate (CR) are derived as insights. Analytical results indicate that i) the main influence of SIC order on the SR and CR lies in the high-SNR power offsets; ii) ISAC provides more degrees of freedom than frequency-division S\&C (FDSAC). Numerical results show that the SR-CR region of ISAC entirely covers that of FDSAC.

[10]  arXiv:2302.03670 [pdf, ps, other]
Title: Private Read Update Write (PRUW) With Heterogeneous Databases
Subjects: Information Theory (cs.IT); Cryptography and Security (cs.CR); Networking and Internet Architecture (cs.NI); Signal Processing (eess.SP)

We investigate the problem of private read update write (PRUW) with heterogeneous storage constrained databases in federated submodel learning (FSL). In FSL a machine learning (ML) model is divided into multiple submodels based on different types of data used to train it. A given user downloads, updates and uploads the updates back to a single submodel of interest, based on the type of user's local data. With PRUW, the process of reading (downloading) and writing (uploading) is carried out such that information theoretic privacy of the updating submodel index and the values of updates is guaranteed. We consider the practical scenario where the submodels are stored in databases with arbitrary (heterogeneous) storage constraints, and provide a PRUW scheme with a storage mechanism that utilizes submodel partitioning and encoding to minimize the communication cost.

Cross-lists for Wed, 8 Feb 23

[11]  arXiv:2302.03121 (cross-list from math.CO) [pdf, other]
Title: Value distributions of perfect nonlinear functions
Comments: 28 pages
Subjects: Combinatorics (math.CO); Information Theory (cs.IT)

In this paper, we study the value distributions of perfect nonlinear functions, i.e., we investigate the sizes of image and preimage sets. Using purely combinatorial tools, we develop a framework that deals with perfect nonlinear functions in the most general setting, generalizing several results that were achieved under specific constraints. For the particularly interesting elementary abelian case, we derive several new strong conditions and classification results on the value distributions. Moreover, we show that most of the classical constructions of perfect nonlinear functions have very specific value distributions, in the sense that they are almost balanced. Consequently, we completely determine the possible value distributions of vectorial Boolean bent functions with output dimension at most 4. Finally, using the discrete Fourier transform, we show that in some cases value distributions can be used to determine whether a given function is perfect nonlinear, or to decide whether given perfect nonlinear functions are equivalent.

[12]  arXiv:2302.03380 (cross-list from cs.LG) [pdf, ps, other]
Title: Phase Transitions in the Detection of Correlated Databases
Comments: 22 pages
Subjects: Machine Learning (cs.LG); Information Theory (cs.IT); Statistics Theory (math.ST)

We study the problem of detecting the correlation between two Gaussian databases $\mathsf{X}\in\mathbb{R}^{n\times d}$ and $\mathsf{Y}^{n\times d}$, each composed of $n$ users with $d$ features. This problem is relevant in the analysis of social media, computational biology, etc. We formulate this as a hypothesis testing problem: under the null hypothesis, these two databases are statistically independent. Under the alternative, however, there exists an unknown permutation $\sigma$ over the set of $n$ users (or, row permutation), such that $\mathsf{X}$ is $\rho$-correlated with $\mathsf{Y}^\sigma$, a permuted version of $\mathsf{Y}$. We determine sharp thresholds at which optimal testing exhibits a phase transition, depending on the asymptotic regime of $n$ and $d$. Specifically, we prove that if $\rho^2d\to0$, as $d\to\infty$, then weak detection (performing slightly better than random guessing) is statistically impossible, irrespectively of the value of $n$. This compliments the performance of a simple test that thresholds the sum all entries of $\mathsf{X}^T\mathsf{Y}$. Furthermore, when $d$ is fixed, we prove that strong detection (vanishing error probability) is impossible for any $\rho<\rho^\star$, where $\rho^\star$ is an explicit function of $d$, while weak detection is again impossible as long as $\rho^2d\to0$. These results close significant gaps in current recent related studies.

[13]  arXiv:2302.03390 (cross-list from cs.LG) [pdf, other]
Title: Learning Discretized Neural Networks under Ricci Flow
Comments: arXiv admin note: text overlap with arXiv:2111.08410
Subjects: Machine Learning (cs.LG); Information Theory (cs.IT); Neural and Evolutionary Computing (cs.NE)

In this paper, we consider Discretized Neural Networks (DNNs) consisting of low-precision weights and activations, which suffer from either infinite or zero gradients caused by the non-differentiable discrete function in the training process. In this case, most training-based DNNs use the standard Straight-Through Estimator (STE) to approximate the gradient w.r.t. discrete value. However, the standard STE will cause the gradient mismatch problem, i.e., the approximated gradient direction may deviate from the steepest descent direction. In other words, the gradient mismatch implies the approximated gradient with perturbations. To address this problem, we introduce the duality theory to regard the perturbation of the approximated gradient as the perturbation of the metric in Linearly Nearly Euclidean (LNE) manifolds. Simultaneously, under the Ricci-DeTurck flow, we prove the dynamical stability and convergence of the LNE metric with the $L^2$-norm perturbation, which can provide a theoretical solution for the gradient mismatch problem. In practice, we also present the steepest descent gradient flow for DNNs on LNE manifolds from the viewpoints of the information geometry and mirror descent. The experimental results on various datasets demonstrate that our method achieves better and more stable performance for DNNs than other representative training-based methods.

[14]  arXiv:2302.03421 (cross-list from stat.ML) [pdf, ps, other]
Title: A unified recipe for deriving (time-uniform) PAC-Bayes bounds
Subjects: Machine Learning (stat.ML); Information Theory (cs.IT); Machine Learning (cs.LG); Statistics Theory (math.ST)

We present a unified framework for deriving PAC-Bayesian generalization bounds. Unlike most previous literature on this topic, our bounds are anytime-valid (i.e., time-uniform), meaning that they hold at all stopping times, not only for a fixed sample size. Our approach combines four tools in the following order: (a) nonnegative supermartingales or reverse submartingales, (b) the method of mixtures, (c) the Donsker-Varadhan formula (or other convex duality principles), and (d) Ville's inequality. We derive time-uniform generalizations of well-known classical PAC-Bayes bounds, such as those of Seeger, McAllester, Maurer, and Catoni, in addition to many recent bounds. We also present several novel bounds and, more importantly, general techniques for constructing them. Despite being anytime-valid, our extensions remain as tight as their fixed-time counterparts. Moreover, they enable us to relax traditional assumptions; in particular, we consider nonstationary loss functions and non-i.i.d. data. In sum, we unify the derivation of past bounds and ease the search for future bounds: one may simply check if our supermartingale or submartingale conditions are met and, if so, be guaranteed a (time-uniform) PAC-Bayes bound.

[15]  arXiv:2302.03559 (cross-list from cs.NI) [pdf, other]
Title: Joint Metrics for EMF Exposure and Coverage in Real-World Homogeneous and Inhomogeneous Cellular Networks
Comments: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible
Subjects: Networking and Internet Architecture (cs.NI); Information Theory (cs.IT)

This paper evaluates the downlink performance of cellular networks in terms of coverage and electromagnetic field (EMF) exposure, in the framework of stochastic geometry. On the one hand, performance metrics are calculated for $\beta$-Ginibre point processes which are shown to faithfully model a large number of motion-invariant networks. On the other hand, performance metrics are derived for inhomogeneous Poisson point processes with a radial intensity measure, which are shown to be a good approximation for motion-variant networks. For both cases, joint and marginal distributions of the EMF exposure and the coverage, and the first moments of the EMF exposure are provided and validated by Monte Carlo simulations using realistic sets of parameters from two urban cellular networks, i.e., 5G NR 2100 (Paris, France) and LTE 1800 (Brussels, Belgium) datasets. In addition, this paper includes the analysis of the impact of the network parameters and discusses the achievable trade-off between coverage and EMF exposure. (This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.)

[16]  arXiv:2302.03577 (cross-list from math.FA) [pdf, ps, other]
Title: Compressed sensing for inverse problems and the sample complexity of the sparse Radon transform
Comments: 60 pages
Subjects: Functional Analysis (math.FA); Information Theory (cs.IT); Optimization and Control (math.OC)

Compressed sensing allows for the recovery of sparse signals from few measurements, whose number is proportional to the sparsity of the unknown signal, up to logarithmic factors. The classical theory typically considers either random linear measurements or subsampled isometries and has found many applications, including accelerated magnetic resonance imaging, which is modeled by the subsampled Fourier transform. In this work, we develop a general theory of infinite-dimensional compressed sensing for abstract inverse problems, possibly ill-posed, involving an arbitrary forward operator. This is achieved by considering a generalized restricted isometry property, and a quasi-diagonalization property of the forward map.
As a notable application, for the first time, we obtain rigorous recovery estimates for the sparse Radon transform (i.e., with a finite number of angles $\theta_1,\dots,\theta_m$), which models computed tomography. In the case when the unknown signal is $s$-sparse with respect to an orthonormal basis of compactly supported wavelets, we prove exact recovery under the condition \[ m\gtrsim s, \] up to logarithmic factors.

[17]  arXiv:2302.03658 (cross-list from cs.DS) [pdf, ps, other]
Title: Planted Bipartite Graph Detection
Comments: 30 pages
Subjects: Data Structures and Algorithms (cs.DS); Information Theory (cs.IT); Machine Learning (cs.LG); Statistics Theory (math.ST)

We consider the task of detecting a hidden bipartite subgraph in a given random graph. Specifically, under the null hypothesis, the graph is a realization of an Erd\H{o}s-R\'{e}nyi random graph over $n$ vertices with edge density $q$. Under the alternative, there exists a planted $k_{\mathsf{R}} \times k_{\mathsf{L}}$ bipartite subgraph with edge density $p>q$. We derive asymptotically tight upper and lower bounds for this detection problem in both the dense regime, where $q,p = \Theta\left(1\right)$, and the sparse regime where $q,p = \Theta\left(n^{-\alpha}\right), \alpha \in \left(0,2\right]$. Moreover, we consider a variant of the above problem, where one can only observe a relatively small part of the graph, by using at most $\mathsf{Q}$ edge queries. For this problem, we derive upper and lower bounds in both the dense and sparse regimes.

[18]  arXiv:2302.03682 (cross-list from math.ST) [pdf, other]
Title: Approximate message passing from random initialization with applications to $\mathbb{Z}_{2}$ synchronization
Subjects: Statistics Theory (math.ST); Information Theory (cs.IT); Signal Processing (eess.SP); Machine Learning (stat.ML)

This paper is concerned with the problem of reconstructing an unknown rank-one matrix with prior structural information from noisy observations. While computing the Bayes-optimal estimator seems intractable in general due to its nonconvex nature, Approximate Message Passing (AMP) emerges as an efficient first-order method to approximate the Bayes-optimal estimator. However, the theoretical underpinnings of AMP remain largely unavailable when it starts from random initialization, a scheme of critical practical utility. Focusing on a prototypical model called $\mathbb{Z}_{2}$ synchronization, we characterize the finite-sample dynamics of AMP from random initialization, uncovering its rapid global convergence. Our theory provides the first non-asymptotic characterization of AMP in this model without requiring either an informative initialization (e.g., spectral initialization) or sample splitting.

Replacements for Wed, 8 Feb 23

[19]  arXiv:2009.14814 (replaced) [pdf, ps, other]
Title: An Upper Bound on Secret Key Rates for General Multiterminal Wiretap Channels
Subjects: Information Theory (cs.IT)
[20]  arXiv:2207.06917 (replaced) [pdf, ps, other]
Title: Online Bayesian Meta-Learning for Cognitive Tracking Radar
Comments: 14 pages, 5 figures
Subjects: Information Theory (cs.IT); Machine Learning (cs.LG); Signal Processing (eess.SP); Systems and Control (eess.SY)
[21]  arXiv:2210.08616 (replaced) [pdf, other]
Title: LoS MIMO-Arrays vs. LoS MIMO-Surfaces
Comments: IEEE EuCAP 2023
Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)
[22]  arXiv:2210.08619 (replaced) [pdf, other]
Title: Modeling the Mutual Coupling of Reconfigurable Metasurfaces
Comments: IEEE EuCAP 2023
Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)
[23]  arXiv:2210.11350 (replaced) [pdf, other]
Title: A Survey on Over-the-Air Computation
Comments: 32 pages, 6 figures; Comments are welcome! (IEEE Communications Surveys & Tutorials)
Subjects: Information Theory (cs.IT); Artificial Intelligence (cs.AI); Signal Processing (eess.SP)
[24]  arXiv:2301.12804 (replaced) [pdf, ps, other]
Title: From ORAN to Cell-Free RAN: Architecture, Performance Analysis, Testbeds and Trials
Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)
[25]  arXiv:2302.01314 (replaced) [pdf, ps, other]
Title: Universal Coding for Shannon Ciphers under Side-Channel Attacks
Comments: 6 pages, 3 figures. previous version has some mistake on the problem set up and the order of the authors. We correct those mistakes in this version. The problem set up in this paper is the same as the one proposed by Santoso and Oohama (Entropy 2019) but is different from the one proposed by Oohama and Santoso (ISIT 2022, arXivarXiv:2201.11670). arXiv admin note: substantial text overlap with arXiv:1801.02563, arXiv:2201.11670, arXiv:1901.05940
Subjects: Information Theory (cs.IT)
[26]  arXiv:2207.00799 (replaced) [pdf, other]
Title: Cramér-Rao Bounds of Near-Field Positioning Based on Electromagnetic Propagation Model
Comments: 16 pages, 8 figures, submitted to IEEE
Subjects: Signal Processing (eess.SP); Information Theory (cs.IT)
[ total of 26 entries: 1-26 ]
[ showing up to 2000 entries per page: fewer | more ]

Disable MathJax (What is MathJax?)

Links to: arXiv, form interface, find, cs, recent, 2302, contact, help  (Access key information)