Information Theory
New submissions
[ showing up to 2000 entries per page: fewer  more ]
New submissions for Wed, 8 Feb 23
 [1] arXiv:2302.03081 [pdf, ps, other]

Title: Permutation resemblanceComments: 12 pagesSubjects: Information Theory (cs.IT); Combinatorics (math.CO); Number Theory (math.NT)
Motivated by the problem of constructing bijective maps with low differential uniformity, we introduce the notion of permutation resemblance of a function, which looks to measure the distance a given map is from being a permutation. We prove several results concerning permutation resemblance and show how it can be used to produce low differentially uniform bijections. We also study the permutation resemblance of planar functions, which over fields of odd characteristic are known not to be bijections and to have the optimal differential uniformity.
 [2] arXiv:2302.03188 [pdf, ps, other]

Title: Stacked Intelligent Metasurfaces Enabled Multiuser Beamforming in the Wave DomainComments: 6 pages, 4 figures, accepted by IEEE ICC 2023Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)
Reconfigurable intelligent surface has recently emerged as a promising technology for reshaping the wireless environment by leveraging massive lowcost passive elements. Prior works mainly focus on a singlelayer metasurface that lacks the capability of suppressing interuser interference. By contrast, we propose in this paper a stacked intelligent metasurfaces (SIM)enabled transceiver design for multiuser multipleinput singleoutput downlink communications. Specifically, a SIM having a multilayer structure is deployed at the base station to perform the transmit beamforming directly in the electromagnetic wave domain. As a result, the conventional digital beamforming and highresolution analogtodigital converters as well as the excessive number of radiofrequency chains are fully removed, which sharply reduces the hardware cost and energy consumption, while substantially decreasing the precoding delay benefiting from the computation at the speed of light. To this end, we formulate an optimization problem for maximizing the sum rate of all users by jointly designing the transmit power allocated to different users and the wavebased beamforming. Finally, numerical results based on a customized alternating optimization algorithm corroborate the effectiveness of our SIMenabled wavebased beamforming design as compared to various benchmark schemes. Most notably, the wavebased beamforming is capable of decreasing the precoding delay by eight orders of magnitude compared to its digital counterpart.
 [3] arXiv:2302.03306 [pdf, other]

Title: Mismatched estimation of nonsymmetric rankone matrices corrupted by structured noiseSubjects: Information Theory (cs.IT); Machine Learning (cs.LG); Statistics Theory (math.ST)
We study the performance of a Bayesian statistician who estimates a rankone signal corrupted by nonsymmetric rotationally invariant noise with a generic distribution of singular values. As the signaltonoise ratio and the noise structure are unknown, a Gaussian setup is incorrectly assumed. We derive the exact analytic expression for the error of the mismatched Bayes estimator and also provide the analysis of an approximate message passing (AMP) algorithm. The first result exploits the asymptotic behavior of spherical integrals for rectangular matrices and of lowrank matrix perturbations; the second one relies on the design and analysis of an auxiliary AMP. The numerical experiments show that there is a performance gap between the AMP and Bayes estimators, which is due to the incorrect estimation of the signal norm.
 [4] arXiv:2302.03335 [pdf, ps, other]

Title: LowLatency Communication using DelayAware Relays Against Reactive AdversariesComments: 30 pagesSubjects: Information Theory (cs.IT)
This work addresses a reactive jamming attack on the lowlatency messages of a victim, wherein the jammer deploys countermeasure detection mechanisms to change its strategy. We highlight that the existing schemes against reactive jammers use relays with instantaneous fullduplex (FD) radios to evade the attack. However, due to the limitation of the radio architecture of the FD helper, instantaneous forwarding may not be possible in practice, thereby leading to increased decoding complexity at the destination and a high detection probability at the adversary. Pointing at this drawback, we propose a delayaware cooperative framework wherein the victim seeks assistance from a delayaware FD helper to forward its messages to the destination within the latency constraints. In particular, we first model the processing delay at the helper based on its hardware architecture, and then propose two lowcomplexity mitigation schemes, wherein the victim and the helper share their uplink frequencies using appropriate energysplitting factors. For both the schemes, we solve the optimization problems of computing the nearoptimal energysplitting factors that minimize the joint error rates at the destination. Finally, through analytical and simulation results, we show that the proposed schemes facilitate the victim in evading the jamming attack whilst deceiving the reactive adversary.
 [5] arXiv:2302.03452 [pdf, other]

Title: CacheAided Communication Schemes via Combinatorial Designs and their $q$analogsSubjects: Information Theory (cs.IT)
We consider the standard broadcast setup with a single server broadcasting information to a number of clients, each of which contains local storage (called cache) of some size, which can store some parts of the available files at the server. The centralized coded caching framework, consists of a caching phase and a delivery phase, both of which are carefully designed in order to use the cache and the channel together optimally. In prior literature, various combinatorial structures have been used to construct coded caching schemes. One of the chief drawbacks of many of these existing constructions is the large subpacketization level, which denotes the number of times a file should be split for the schemes to provide coding gain. In this work, using a new binary matrix model, we present several novel constructions for coded caching based on the various types of combinatorial designs and their $q$analogs, which are also called subspace designs. While most of the schemes constructed in this work (based on existing designs) have a high cache requirement, they provide a rate that is either constant or decreasing, and moreover require competitively small levels of subpacketization, which is an extremely important feature in practical applications of coded caching. We also apply our constructions to the distributed computing framework of MapReduce, which consists of three phases, the Map phase, the Shuffle phase and the Reduce phase. Using our binary matrix framework, we present a new simple generic coded data shuffling scheme. Employing our designsbased constructions in conjunction with this new shuffling scheme, we obtain new coded computing schemes which have low file complexity, with marginally higher communication load compared to the optimal scheme for equivalent parameters. We show that our schemes can neatly extend to the scenario with full and partial stragglers also.
 [6] arXiv:2302.03524 [pdf, other]

Title: Multiple Keycast over NetworksSubjects: Information Theory (cs.IT)
The multicast keydissemination problem over noiseless networks, introduced by Langberg and Effros [ITW 2022], here called the ``keycast'' problem, captures the task of disseminating a shared secret random key to a set of terminals over a given network. Unlike traditional communication, where messages must be delivered from source to destination(s) unchanged, keycast is more flexible since keycast need not require source reconstruction at destination nodes. For example, the distributed keys can be mixtures of sources from which the sources themselves may be unrecoverable.
The work at hand considers key dissemination in the singlesource, multiplemulticast network coding setting, i.e., the ``multiple keycast'' problem. Here, distinct keys are to be simultaneously transmitted from a single source node to multiple terminal sets, one shared random key per multicast set. Scenarios include the secure setting, in which only the source and intended destinations gain information about a given key; and the nonsecure setting in which the only requirement is that the knowledge of one key does not reveal information about another. In both settings, we present combinatorial conditions for key dissemination and design corresponding multiple keycast schemes. In addition, we compare the multiple keycast rate with and without the restriction of source reconstruction, the former corresponding to traditional forms of communication; keycast achieves a strict advantage in rate when source reconstruction is relaxed.  [7] arXiv:2302.03549 [pdf, other]

Title: An Information Bottleneck Approach to SecrecyPreserving Gaussian Mixture ClassificationSubjects: Information Theory (cs.IT)
The information bottleneck (IB) approach, initially introduced by [1] to assess the compressionrelevance tradeoff for a remote source coding problem in communication, quickly gains popularity recently in its application to modern machine learning (ML). Unlike the use of most  if not all  IB in the literature, either for the analysis of, say deep neural networks, or as an optimization objective, in this paper, we propose to address the secrecy issue in ML, by considering the fundamental model of Gaussian mixture classification. We derive, for the first time, closedform achievable bounds for the IB problem under the above setting, and provide precise characterization of the underlying performancesecrecy tradeoff. Experiments on both synthetic and realworld data are performed to confirm the satisfactory performance of the proposed scheme.
 [8] arXiv:2302.03574 [pdf, other]

Title: A Dominant Interferer plus Mean Fieldbased Approximation for SINR Meta Distribution in Wireless NetworksSubjects: Information Theory (cs.IT); Systems and Control (eess.SY)
This paper proposes a novel approach for computing the meta distribution of the signaltointerferenceplusnoise ratio (SINR) for the downlink transmission in a wireless network with Rayleigh fading. The novel approach relies on an approximation mix of exact and meanfield analysis of interference (dominant interfererbased approximation) to reduce the complexity of analysis and enhance tractability. In particular, the proposed approximation omits the need to compute the first or the second moment of the SINR that is used in the beta approximation typically adopted in the literature but requires of computing the joint distance distributions. We first derive the proposed approximation based on a Poisson point process (PPP) network with a standard pathloss and Rayleigh fading and then illustrate its accuracy and operability in another four widely used point processes: Poisson bipolar network, Mat\'{e}rn cluster process (MCP), $K$tier PPP and Poisson line Cox process (PLCP). Specifically, we obtain the SINR meta distribution for PLCP networks for the first time. Even though the proposed approximation looks simple but it shows good matching in comparison to the popular beta approximation as well as the MonteCarlo simulations, which opens the door to adopting this approximation in more advanced network architectures.
 [9] arXiv:2302.03656 [pdf, ps, other]

Title: Revealing the Impact of SIC in NOMAISACComments: 5 pagesSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
The impact of successive interference cancellation (SIC) in nonorthogonal multiple access integrated sensing and communications (NOMAISAC) is analyzed. A twostage SICbased framework is proposed to deal with the intercommunication user and interfunctionality interferences. The performance of sensing and communications (S\&C) is analyzed for two SIC orders, i.e., the communicationscentric SIC and the sensingcentric SIC. For each design, diversity orders, high signaltonoise ratio (SNR) slopes, and highSNR power offsets of the sensing rate (SR) and communication rate (CR) are derived as insights. Analytical results indicate that i) the main influence of SIC order on the SR and CR lies in the highSNR power offsets; ii) ISAC provides more degrees of freedom than frequencydivision S\&C (FDSAC). Numerical results show that the SRCR region of ISAC entirely covers that of FDSAC.
 [10] arXiv:2302.03670 [pdf, ps, other]

Title: Private Read Update Write (PRUW) With Heterogeneous DatabasesSubjects: Information Theory (cs.IT); Cryptography and Security (cs.CR); Networking and Internet Architecture (cs.NI); Signal Processing (eess.SP)
We investigate the problem of private read update write (PRUW) with heterogeneous storage constrained databases in federated submodel learning (FSL). In FSL a machine learning (ML) model is divided into multiple submodels based on different types of data used to train it. A given user downloads, updates and uploads the updates back to a single submodel of interest, based on the type of user's local data. With PRUW, the process of reading (downloading) and writing (uploading) is carried out such that information theoretic privacy of the updating submodel index and the values of updates is guaranteed. We consider the practical scenario where the submodels are stored in databases with arbitrary (heterogeneous) storage constraints, and provide a PRUW scheme with a storage mechanism that utilizes submodel partitioning and encoding to minimize the communication cost.
Crosslists for Wed, 8 Feb 23
 [11] arXiv:2302.03121 (crosslist from math.CO) [pdf, other]

Title: Value distributions of perfect nonlinear functionsComments: 28 pagesSubjects: Combinatorics (math.CO); Information Theory (cs.IT)
In this paper, we study the value distributions of perfect nonlinear functions, i.e., we investigate the sizes of image and preimage sets. Using purely combinatorial tools, we develop a framework that deals with perfect nonlinear functions in the most general setting, generalizing several results that were achieved under specific constraints. For the particularly interesting elementary abelian case, we derive several new strong conditions and classification results on the value distributions. Moreover, we show that most of the classical constructions of perfect nonlinear functions have very specific value distributions, in the sense that they are almost balanced. Consequently, we completely determine the possible value distributions of vectorial Boolean bent functions with output dimension at most 4. Finally, using the discrete Fourier transform, we show that in some cases value distributions can be used to determine whether a given function is perfect nonlinear, or to decide whether given perfect nonlinear functions are equivalent.
 [12] arXiv:2302.03380 (crosslist from cs.LG) [pdf, ps, other]

Title: Phase Transitions in the Detection of Correlated DatabasesComments: 22 pagesSubjects: Machine Learning (cs.LG); Information Theory (cs.IT); Statistics Theory (math.ST)
We study the problem of detecting the correlation between two Gaussian databases $\mathsf{X}\in\mathbb{R}^{n\times d}$ and $\mathsf{Y}^{n\times d}$, each composed of $n$ users with $d$ features. This problem is relevant in the analysis of social media, computational biology, etc. We formulate this as a hypothesis testing problem: under the null hypothesis, these two databases are statistically independent. Under the alternative, however, there exists an unknown permutation $\sigma$ over the set of $n$ users (or, row permutation), such that $\mathsf{X}$ is $\rho$correlated with $\mathsf{Y}^\sigma$, a permuted version of $\mathsf{Y}$. We determine sharp thresholds at which optimal testing exhibits a phase transition, depending on the asymptotic regime of $n$ and $d$. Specifically, we prove that if $\rho^2d\to0$, as $d\to\infty$, then weak detection (performing slightly better than random guessing) is statistically impossible, irrespectively of the value of $n$. This compliments the performance of a simple test that thresholds the sum all entries of $\mathsf{X}^T\mathsf{Y}$. Furthermore, when $d$ is fixed, we prove that strong detection (vanishing error probability) is impossible for any $\rho<\rho^\star$, where $\rho^\star$ is an explicit function of $d$, while weak detection is again impossible as long as $\rho^2d\to0$. These results close significant gaps in current recent related studies.
 [13] arXiv:2302.03390 (crosslist from cs.LG) [pdf, other]

Title: Learning Discretized Neural Networks under Ricci FlowComments: arXiv admin note: text overlap with arXiv:2111.08410Subjects: Machine Learning (cs.LG); Information Theory (cs.IT); Neural and Evolutionary Computing (cs.NE)
In this paper, we consider Discretized Neural Networks (DNNs) consisting of lowprecision weights and activations, which suffer from either infinite or zero gradients caused by the nondifferentiable discrete function in the training process. In this case, most trainingbased DNNs use the standard StraightThrough Estimator (STE) to approximate the gradient w.r.t. discrete value. However, the standard STE will cause the gradient mismatch problem, i.e., the approximated gradient direction may deviate from the steepest descent direction. In other words, the gradient mismatch implies the approximated gradient with perturbations. To address this problem, we introduce the duality theory to regard the perturbation of the approximated gradient as the perturbation of the metric in Linearly Nearly Euclidean (LNE) manifolds. Simultaneously, under the RicciDeTurck flow, we prove the dynamical stability and convergence of the LNE metric with the $L^2$norm perturbation, which can provide a theoretical solution for the gradient mismatch problem. In practice, we also present the steepest descent gradient flow for DNNs on LNE manifolds from the viewpoints of the information geometry and mirror descent. The experimental results on various datasets demonstrate that our method achieves better and more stable performance for DNNs than other representative trainingbased methods.
 [14] arXiv:2302.03421 (crosslist from stat.ML) [pdf, ps, other]

Title: A unified recipe for deriving (timeuniform) PACBayes boundsSubjects: Machine Learning (stat.ML); Information Theory (cs.IT); Machine Learning (cs.LG); Statistics Theory (math.ST)
We present a unified framework for deriving PACBayesian generalization bounds. Unlike most previous literature on this topic, our bounds are anytimevalid (i.e., timeuniform), meaning that they hold at all stopping times, not only for a fixed sample size. Our approach combines four tools in the following order: (a) nonnegative supermartingales or reverse submartingales, (b) the method of mixtures, (c) the DonskerVaradhan formula (or other convex duality principles), and (d) Ville's inequality. We derive timeuniform generalizations of wellknown classical PACBayes bounds, such as those of Seeger, McAllester, Maurer, and Catoni, in addition to many recent bounds. We also present several novel bounds and, more importantly, general techniques for constructing them. Despite being anytimevalid, our extensions remain as tight as their fixedtime counterparts. Moreover, they enable us to relax traditional assumptions; in particular, we consider nonstationary loss functions and noni.i.d. data. In sum, we unify the derivation of past bounds and ease the search for future bounds: one may simply check if our supermartingale or submartingale conditions are met and, if so, be guaranteed a (timeuniform) PACBayes bound.
 [15] arXiv:2302.03559 (crosslist from cs.NI) [pdf, other]

Title: Joint Metrics for EMF Exposure and Coverage in RealWorld Homogeneous and Inhomogeneous Cellular NetworksAuthors: Quentin Gontier, Charles Wiame, Shanshan Wang, Marco Di Renzo, Joe Wiart, François Horlin, Christo Tsigros, Claude Oestges, Philippe De DonckerComments: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessibleSubjects: Networking and Internet Architecture (cs.NI); Information Theory (cs.IT)
This paper evaluates the downlink performance of cellular networks in terms of coverage and electromagnetic field (EMF) exposure, in the framework of stochastic geometry. On the one hand, performance metrics are calculated for $\beta$Ginibre point processes which are shown to faithfully model a large number of motioninvariant networks. On the other hand, performance metrics are derived for inhomogeneous Poisson point processes with a radial intensity measure, which are shown to be a good approximation for motionvariant networks. For both cases, joint and marginal distributions of the EMF exposure and the coverage, and the first moments of the EMF exposure are provided and validated by Monte Carlo simulations using realistic sets of parameters from two urban cellular networks, i.e., 5G NR 2100 (Paris, France) and LTE 1800 (Brussels, Belgium) datasets. In addition, this paper includes the analysis of the impact of the network parameters and discusses the achievable tradeoff between coverage and EMF exposure. (This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.)
 [16] arXiv:2302.03577 (crosslist from math.FA) [pdf, ps, other]

Title: Compressed sensing for inverse problems and the sample complexity of the sparse Radon transformComments: 60 pagesSubjects: Functional Analysis (math.FA); Information Theory (cs.IT); Optimization and Control (math.OC)
Compressed sensing allows for the recovery of sparse signals from few measurements, whose number is proportional to the sparsity of the unknown signal, up to logarithmic factors. The classical theory typically considers either random linear measurements or subsampled isometries and has found many applications, including accelerated magnetic resonance imaging, which is modeled by the subsampled Fourier transform. In this work, we develop a general theory of infinitedimensional compressed sensing for abstract inverse problems, possibly illposed, involving an arbitrary forward operator. This is achieved by considering a generalized restricted isometry property, and a quasidiagonalization property of the forward map.
As a notable application, for the first time, we obtain rigorous recovery estimates for the sparse Radon transform (i.e., with a finite number of angles $\theta_1,\dots,\theta_m$), which models computed tomography. In the case when the unknown signal is $s$sparse with respect to an orthonormal basis of compactly supported wavelets, we prove exact recovery under the condition \[ m\gtrsim s, \] up to logarithmic factors.  [17] arXiv:2302.03658 (crosslist from cs.DS) [pdf, ps, other]

Title: Planted Bipartite Graph DetectionComments: 30 pagesSubjects: Data Structures and Algorithms (cs.DS); Information Theory (cs.IT); Machine Learning (cs.LG); Statistics Theory (math.ST)
We consider the task of detecting a hidden bipartite subgraph in a given random graph. Specifically, under the null hypothesis, the graph is a realization of an Erd\H{o}sR\'{e}nyi random graph over $n$ vertices with edge density $q$. Under the alternative, there exists a planted $k_{\mathsf{R}} \times k_{\mathsf{L}}$ bipartite subgraph with edge density $p>q$. We derive asymptotically tight upper and lower bounds for this detection problem in both the dense regime, where $q,p = \Theta\left(1\right)$, and the sparse regime where $q,p = \Theta\left(n^{\alpha}\right), \alpha \in \left(0,2\right]$. Moreover, we consider a variant of the above problem, where one can only observe a relatively small part of the graph, by using at most $\mathsf{Q}$ edge queries. For this problem, we derive upper and lower bounds in both the dense and sparse regimes.
 [18] arXiv:2302.03682 (crosslist from math.ST) [pdf, other]

Title: Approximate message passing from random initialization with applications to $\mathbb{Z}_{2}$ synchronizationSubjects: Statistics Theory (math.ST); Information Theory (cs.IT); Signal Processing (eess.SP); Machine Learning (stat.ML)
This paper is concerned with the problem of reconstructing an unknown rankone matrix with prior structural information from noisy observations. While computing the Bayesoptimal estimator seems intractable in general due to its nonconvex nature, Approximate Message Passing (AMP) emerges as an efficient firstorder method to approximate the Bayesoptimal estimator. However, the theoretical underpinnings of AMP remain largely unavailable when it starts from random initialization, a scheme of critical practical utility. Focusing on a prototypical model called $\mathbb{Z}_{2}$ synchronization, we characterize the finitesample dynamics of AMP from random initialization, uncovering its rapid global convergence. Our theory provides the first nonasymptotic characterization of AMP in this model without requiring either an informative initialization (e.g., spectral initialization) or sample splitting.
Replacements for Wed, 8 Feb 23
 [19] arXiv:2009.14814 (replaced) [pdf, ps, other]

Title: An Upper Bound on Secret Key Rates for General Multiterminal Wiretap ChannelsSubjects: Information Theory (cs.IT)
 [20] arXiv:2207.06917 (replaced) [pdf, ps, other]

Title: Online Bayesian MetaLearning for Cognitive Tracking RadarComments: 14 pages, 5 figuresSubjects: Information Theory (cs.IT); Machine Learning (cs.LG); Signal Processing (eess.SP); Systems and Control (eess.SY)
 [21] arXiv:2210.08616 (replaced) [pdf, other]

Title: LoS MIMOArrays vs. LoS MIMOSurfacesComments: IEEE EuCAP 2023Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)
 [22] arXiv:2210.08619 (replaced) [pdf, other]

Title: Modeling the Mutual Coupling of Reconfigurable MetasurfacesComments: IEEE EuCAP 2023Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)
 [23] arXiv:2210.11350 (replaced) [pdf, other]

Title: A Survey on OvertheAir ComputationComments: 32 pages, 6 figures; Comments are welcome! (IEEE Communications Surveys & Tutorials)Subjects: Information Theory (cs.IT); Artificial Intelligence (cs.AI); Signal Processing (eess.SP)
 [24] arXiv:2301.12804 (replaced) [pdf, ps, other]

Title: From ORAN to CellFree RAN: Architecture, Performance Analysis, Testbeds and TrialsAuthors: Yang Cao, Ziyang Zhang, Xinjiang Xia, Pengzhe Xin, Dongjie Liu, Kang Zheng, Mengting Lou, Jing Jin, Qixing Wang, Dongming Wang, Yongming Huang, Xiaohu You, Jiangzhou WangSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
 [25] arXiv:2302.01314 (replaced) [pdf, ps, other]

Title: Universal Coding for Shannon Ciphers under SideChannel AttacksComments: 6 pages, 3 figures. previous version has some mistake on the problem set up and the order of the authors. We correct those mistakes in this version. The problem set up in this paper is the same as the one proposed by Santoso and Oohama (Entropy 2019) but is different from the one proposed by Oohama and Santoso (ISIT 2022, arXivarXiv:2201.11670). arXiv admin note: substantial text overlap with arXiv:1801.02563, arXiv:2201.11670, arXiv:1901.05940Subjects: Information Theory (cs.IT)
 [26] arXiv:2207.00799 (replaced) [pdf, other]

Title: CramérRao Bounds of NearField Positioning Based on Electromagnetic Propagation ModelComments: 16 pages, 8 figures, submitted to IEEESubjects: Signal Processing (eess.SP); Information Theory (cs.IT)
[ showing up to 2000 entries per page: fewer  more ]
Disable MathJax (What is MathJax?)
Links to: arXiv, form interface, find, cs, recent, 2302, contact, help (Access key information)