Information Theory
New submissions
[ showing up to 2000 entries per page: fewer | more ]
New submissions for Mon, 18 Mar 24
- [1] arXiv:2403.10002 [pdf, ps, other]
-
Title: Joint Group Scheduling and Multicast Beamforming for Downlink Large-Scale Multi-Group MulticastComments: 13 pages, 8 figures, 6 tablesSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
Next-generation wireless networks need to handle massive user access effectively. This paper addresses the problem of joint group scheduling and multicast beamforming for downlink multicast with many active groups. Aiming to maximize the minimum user throughput, we propose a three-phase approach to tackle this difficult joint optimization problem efficiently. In Phase 1, we utilize the optimal multicast beamforming structure obtained recently to find the group-channel directions for all groups. We propose two low-complexity scheduling algorithms in Phase 2, which determine the subset of groups in each time slot sequentially and the total number of time slots required for all groups. The first algorithm measures the level of spatial separation among groups and selects the dissimilar groups that maximize the minimum user rate into the same time slot. In contrast, the second algorithm first identifies the spatially correlated groups via a learning-based clustering method based on the group-channel directions, and then separates spatially similar groups into different time slots. Finally, the multicast beamformers for the scheduled groups are obtained in each time slot by a computationally efficient method. Simulation results show that our proposed approaches can effectively capture the level of spatial separation among groups for scheduling to improve the minimum user throughput over the conventional approach that serves all groups in a single time slot or one group per time slot, and can be executed with low computational complexity.
- [2] arXiv:2403.10089 [pdf, other]
-
Title: Approximation and bounding techniques for the Fisher-Rao distancesAuthors: Frank NielsenComments: 38 pagesSubjects: Information Theory (cs.IT); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
The Fisher-Rao distance between two probability distributions of a statistical model is defined as the Riemannian geodesic distance induced by the Fisher information metric. In order to calculate the Fisher-Rao distance in closed-form, we need (1) to elicit a formula for the Fisher-Rao geodesics, and (2) to integrate the Fisher length element along those geodesics. We consider several numerically robust approximation and bounding techniques for the Fisher-Rao distances: First, we report generic upper bounds on Fisher-Rao distances based on closed-form 1D Fisher-Rao distances of submodels. Second, we describe several generic approximation schemes depending on whether the Fisher-Rao geodesics or pregeodesics are available in closed-form or not. In particular, we obtain a generic method to guarantee an arbitrarily small additive error on the approximation provided that Fisher-Rao pregeodesics and tight lower and upper bounds are available. Third, we consider the case of Fisher metrics being Hessian metrics, and report generic tight upper bounds on the Fisher-Rao distances using techniques of information geometry. Uniparametric and biparametric statistical models always have Fisher Hessian metrics, and in general a simple test allows to check whether the Fisher information matrix yields a Hessian metric or not. Fourth, we consider elliptical distribution families and show how to apply the above techniques to these models. We also propose two new distances based either on the Fisher-Rao lengths of curves serving as proxies of Fisher-Rao geodesics, or based on the Birkhoff/Hilbert projective cone distance. Last, we consider an alternative group-theoretic approach for statistical transformation models based on the notion of maximal invariant which yields insights on the structures of the Fisher-Rao distance formula which may be used fruitfully in applications.
- [3] arXiv:2403.10230 [pdf, ps, other]
-
Title: Fairness Optimization for Intelligent Reflecting Surface Aided Uplink Rate-Splitting Multiple AccessComments: This work has been submitted to TCOMSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
This paper studies the fair transmission design for an intelligent reflecting surface (IRS) aided rate-splitting multiple access (RSMA). IRS is used to establish a good signal propagation environment and enhance the RSMA transmission performance. The fair rate adaption problem is constructed as a max-min optimization problem. To solve the optimization problem, we adopt an alternative optimization (AO) algorithm to optimize the power allocation, beamforming, and decoding order, respectively. A generalized power iteration (GPI) method is proposed to optimize the receive beamforming, which can improve the minimum rate of devices and reduce the optimization complexity. At the base station (BS), a successive group decoding (SGD) algorithm is proposed to tackle the uplink signal estimation, which trades off the fairness and complexity of decoding. At the same time, we also consider robust communication with imperfect channel state information at the transmitter (CSIT), which studies robust optimization by using lower bound expressions on the expected data rates. Extensive numerical results show that the proposed optimization algorithm can significantly improve the performance of fairness. It also provides reliable results for uplink communication with imperfect CSIT.
- [4] arXiv:2403.10232 [pdf, other]
-
Title: Matrix Completion via Nonsmooth Regularization of Fully Connected Neural NetworksSubjects: Information Theory (cs.IT); Machine Learning (cs.LG)
Conventional matrix completion methods approximate the missing values by assuming the matrix to be low-rank, which leads to a linear approximation of missing values. It has been shown that enhanced performance could be attained by using nonlinear estimators such as deep neural networks. Deep fully connected neural networks (FCNNs), one of the most suitable architectures for matrix completion, suffer from over-fitting due to their high capacity, which leads to low generalizability. In this paper, we control over-fitting by regularizing the FCNN model in terms of the $\ell_{1}$ norm of intermediate representations and nuclear norm of weight matrices. As such, the resulting regularized objective function becomes nonsmooth and nonconvex, i.e., existing gradient-based methods cannot be applied to our model. We propose a variant of the proximal gradient method and investigate its convergence to a critical point. In the initial epochs of FCNN training, the regularization terms are ignored, and through epochs, the effect of that increases. The gradual addition of nonsmooth regularization terms is the main reason for the better performance of the deep neural network with nonsmooth regularization terms (DNN-NSR) algorithm. Our simulations indicate the superiority of the proposed algorithm in comparison with existing linear and nonlinear algorithms.
- [5] arXiv:2403.10307 [pdf, other]
-
Title: Chernoff Information as a Privacy Constraint for Adversarial ClassificationSubjects: Information Theory (cs.IT)
This work studies a privacy metric based on Chernoff information, \textit{Chernoff differential privacy}, due to its significance in characterization of classifier performance. Adversarial classification, as any other classification problem is built around minimization of the (average or correct detection) probability of error in deciding on either of the classes in the case of binary classification. Unlike the classical hypothesis testing problem, where the false alarm and mis-detection probabilities are handled separately resulting in an asymmetric behavior of the best error exponent, in this work, we focus on the Bayesian setting and characterize the relationship between the best error exponent of the average error probability and $\varepsilon-$differential privacy. Accordingly, we re-derive Chernoff differential privacy in terms of $\varepsilon-$differential privacy using the Radon-Nikodym derivative and show that it satisfies the composition property. Subsequently, we present numerical evaluation results, which demonstrates that Chernoff information outperforms Kullback-Leibler divergence as a function of the privacy parameter $\varepsilon$, the impact of the adversary's attack and global sensitivity for the problem of adversarial classification in Laplace mechanisms.
- [6] arXiv:2403.10412 [pdf, other]
-
Title: RIS-Assisted Physical Layer Security in Emerging RF and Optical Wireless Communication Systems: A Comprehensive SurveyAuthors: Majid H. Khoshafa, Omar Maraqa, Jules M. Moualeu, Sylvester Aboagye, Telex M. N. Ngatched, Mohamed H. Ahmed, Yasser Gadallah, Marco Di RenzoComments: This work has been submitted to the IEEE for possible publicationSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
Physical layer security (PLS) has received a growing interest from the research community for its ability to safeguard data confidentiality without relying on key distribution or encryption/decryption. However, the evolution towards the 5G technology and beyond poses new security challenges that must be addressed in order to fulfill the unprecedented performance requirements of future wireless networks. Among the potential enabling technologies, RIS has attracted extensive attention due to its ability to proactively and intelligently reconfigure the wireless propagation environment to combat dynamic wireless channel impairments. Consequently, the RIS technology can be adopted to improve the information-theoretic security of both RF and OWC systems. This survey paper provides a comprehensive overview of the information-theoretic security of RIS-based RF and optical systems. The article first discusses the fundamental concepts of PLS and RIS technologies, followed by their combination in both RF and OWC systems. Subsequently, some optimization techniques are presented in the context of the underlying system model, followed by an assessment of the impact of RIS-assisted PLS through a comprehensive performance analysis. Given that the computational complexity of future communication systems that adopt RIS-assisted PLS is likely to increase rapidly as the number of interactions between the users and infrastructure grows, ML is seen as a promising approach to address this complexity issue while sustaining or improving the network performance. A discussion of recent research studies on RIS-assisted PLS-based systems embedded with ML is presented. Furthermore, some important open research challenges are proposed and discussed to provide insightful future research directions, with the aim of moving a step closer towards the development and implementation of the forthcoming 6G wireless technology.
Cross-lists for Mon, 18 Mar 24
- [7] arXiv:2403.09958 (cross-list from eess.SP) [pdf, other]
-
Title: Decentralizing Coherent Joint Transmission Precoding via Deterministic EquivalentsSubjects: Signal Processing (eess.SP); Information Theory (cs.IT)
In order to control the inter-cell interference for a multi-cell multi-user multiple-input multiple-output network, we consider the precoder design for coordinated multi-point with downlink coherent joint transmission. To avoid costly information exchange among the cooperating base stations in a centralized precoding scheme, we propose a decentralized one by considering the power minimization problem. By approximating the inter-cell interference using the deterministic equivalents, this problem is decoupled to sub-problems which are solved in a decentralized manner at different base stations. Simulation results demonstrate the effectiveness of our proposed decentralized precoding scheme, where only 2 ~ 7% more transmit power is needed compared with the optimal centralized precoder.
- [8] arXiv:2403.10202 (cross-list from eess.IV) [pdf, other]
-
Title: Learning on JPEG-LDPC Compressed Images: Classifying with SyndromesComments: 5 pages, 3 figures, conference paper, submitted to the EUSIPCO 2024 ConferenceSubjects: Image and Video Processing (eess.IV); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Information Theory (cs.IT); Machine Learning (cs.LG)
In goal-oriented communications, the objective of the receiver is often to apply a Deep-Learning model, rather than reconstructing the original data. In this context, direct learning over compressed data, without any prior decoding, holds promise for enhancing the time-efficient execution of inference models at the receiver. However, conventional entropic-coding methods like Huffman and Arithmetic break data structure, rendering them unsuitable for learning without decoding. In this paper, we propose an alternative approach in which entropic coding is realized with Low-Density Parity Check (LDPC) codes. We hypothesize that Deep Learning models can more effectively exploit the internal code structure of LDPC codes. At the receiver, we leverage a specific class of Recurrent Neural Networks (RNNs), specifically Gated Recurrent Unit (GRU), trained for image classification. Our numerical results indicate that classification based on LDPC-coded bit-planes surpasses Huffman and Arithmetic coding, while necessitating a significantly smaller learning model. This demonstrates the efficiency of classification directly from LDPC-coded data, eliminating the need for any form of decompression, even partial, prior to applying the learning model.
- [9] arXiv:2403.10444 (cross-list from cs.LG) [pdf, other]
-
Title: Optimal Block-Level Draft Verification for Accelerating Speculative DecodingSubjects: Machine Learning (cs.LG); Computation and Language (cs.CL); Data Structures and Algorithms (cs.DS); Information Theory (cs.IT)
Speculative decoding has shown to be an effective method for lossless acceleration of large language models (LLMs) during inference. In each iteration, the algorithm first uses a smaller model to draft a block of tokens. The tokens are then verified by the large model in parallel and only a subset of tokens will be kept to guarantee that the final output follows the distribution of the large model. In all of the prior speculative decoding works, the draft verification is performed token-by-token independently. In this work, we propose a better draft verification algorithm that provides additional wall-clock speedup without incurring additional computation cost and draft tokens. We first formulate the draft verification step as a block-level optimal transport problem. The block-level formulation allows us to consider a wider range of draft verification algorithms and obtain a higher number of accepted tokens in expectation in one draft block. We propose a verification algorithm that achieves the optimal accepted length for the block-level transport problem. We empirically evaluate our proposed block-level verification algorithm in a wide range of tasks and datasets, and observe consistent improvements in wall-clock speedup when compared to token-level verification algorithm. To the best of our knowledge, our work is the first to establish improvement over speculative decoding through a better draft verification algorithm.
Replacements for Mon, 18 Mar 24
- [10] arXiv:2302.01856 (replaced) [pdf, other]
-
Title: Entropy of Exchangeable Random GraphsSubjects: Information Theory (cs.IT); Combinatorics (math.CO); Statistics Theory (math.ST)
- [11] arXiv:2310.17327 (replaced) [pdf, ps, other]
-
Title: Near-Field Positioning and Attitude Sensing Based on Electromagnetic Propagation ModelingComments: 18 pages, 10 figures. JSAC - Special Issue on Positioning and Sensing Over Wireless Networks, Major revisionSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
- [12] arXiv:2311.07041 (replaced) [pdf, other]
-
Title: Deep Joint Source Channel Coding With Attention Modules Over MIMO ChannelsSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
- [13] arXiv:2301.09590 (replaced) [pdf, ps, other]
-
Title: Outer Strong Blocking SetsSubjects: Combinatorics (math.CO); Information Theory (cs.IT)
- [14] arXiv:2305.11471 (replaced) [pdf, ps, other]
-
Title: Concrete Quantum Channels and Algebraic Structure of Abstract Quantum ChannelsAuthors: M. N. N. NamboodiriSubjects: Quantum Physics (quant-ph); Information Theory (cs.IT)
- [15] arXiv:2306.16506 (replaced) [pdf, other]
-
Title: Equivariant Neural Networks for Indirect MeasurementsComments: 23 pages, 7 figuresSubjects: Numerical Analysis (math.NA); Information Theory (cs.IT); Functional Analysis (math.FA)
- [16] arXiv:2309.01237 (replaced) [pdf, other]
-
Title: The Information Geometry of UMAPComments: 11 pages, 2 figures, 3 tables; Github repo (this https URL)Subjects: Computational Geometry (cs.CG); Discrete Mathematics (cs.DM); Information Theory (cs.IT); Geometric Topology (math.GT)
[ showing up to 2000 entries per page: fewer | more ]
Disable MathJax (What is MathJax?)
Links to: arXiv, form interface, find, cs, recent, 2403, contact, help (Access key information)