We gratefully acknowledge support from
the Simons Foundation and member institutions.

Information Theory

New submissions

[ total of 24 entries: 1-24 ]
[ showing up to 2000 entries per page: fewer | more ]

New submissions for Fri, 23 Oct 20

[1]  arXiv:2010.11317 [pdf, other]
Title: Full-Duplex and Dynamic-TDD: Pushing the Limits of Spectrum Reuse in Multi-Cell Communications
Comments: 15 pages, 6 figures. Accepted to IEEE Wireless Communications - Special Issue on Full Duplex Communications Theory, Standardization and Practice
Subjects: Information Theory (cs.IT); Networking and Internet Architecture (cs.NI); Signal Processing (eess.SP)

Although in cellular networks full-duplex and dynamic time-division duplexing promise increased spectrum efficiency, their potential is so far challenged by increased interference. While previous studies have shown that self-interference can be suppressed to a sufficient level, we show that the cross-link interference for both duplexing modes, especially from base station to base station, is the remaining challenge in multi-cell networks, restricting the uplink performance. Using beamforming techniques of low-complexity, we show that this interference can be mitigated, and that full-duplex and dynamic time-division duplexing can substantially increase the capacity of multi-cell networks. Our results suggest that if we can control the cross link interference in full-duplex, then we can almost double the multi cell network capacity as well as user throughput. Therefore, the techniques in this paper have the potentiality to enable a smooth introduction of full-duplex into cellular systems.

[2]  arXiv:2010.11321 [pdf, ps, other]
Title: MRI Image Recovery using Damped Denoising Vector AMP
Subjects: Information Theory (cs.IT)

Motivated by image recovery in magnetic resonance imaging (MRI), we propose a new approach to solving linear inverse problems based on iteratively calling a deep neural-network, sometimes referred to as plug-and-play recovery. Our approach is based on the vector approximate message passing (VAMP) algorithm, which is known for mean-squared error (MSE)-optimal recovery under certain conditions. The forward operator in MRI, however, does not satisfy these conditions, and thus we design new damping and initialization schemes to help VAMP. The resulting DD-VAMP++ algorithm is shown to outperform existing algorithms in convergence speed and accuracy when recovering images from the fastMRI database for the practical case of Cartesian sampling.

[3]  arXiv:2010.11372 [pdf, other]
Title: Symmetrical Z-Complementary Code Sets (SZCCSs) for Optimal Training in Generalized Spatial Modulation
Comments: 13 pages, 7 figures, submitted to IEEE Transactions on Signal Processing
Subjects: Information Theory (cs.IT)

This paper introduces a novel class of code sets, called "symmetrical Z-complementary code sets (SZCCSs)" , whose aperiodic auto- and cross- correlation sums exhibit zero-correlation zones (ZCZs) at both the front-end and tail-end of the entire correlation window. Three constructions of (optimal) SZCCSs based on general Boolean functions are presented. As a second major contribution, we apply SZCCSs to design optimal training sequences for broadband generalized spatial modulation (GSM) systems over frequency-selective channels.
Key words: Complementary code set, channel estimation, training sequence design, generalized spatial modulation, frequency-selective channels.

[4]  arXiv:2010.11380 [pdf, ps, other]
Title: A Hybrid Approach to Coded Compressed Sensing where Coupling Takes Place via the Outer Code
Subjects: Information Theory (cs.IT)

This article seeks to advance coded compressed sensing (CCS) as a practical scheme for unsourced random access. The original CCS algorithm features a concatenated structure where an inner code is tasked with support recovery, and an outer tree code conducts message disambiguation. Recently, a link between CCS and sparse regression codes was established, leading to the application of approximate message passing (AMP) to CCS. This connection was subsequently strengthened by integrating AMP and belief propagation on the outer code through a dynamic denoiser. Along these lines, this work shows how block diagonal sensing matrices akin to those used in traditional CCS, together with the aforementioned dynamic denoiser, form an effective means to get good performance at low-complexity. This novel architecture can be used to scale this scheme to dimensions that were previously impractical. Findings are supported by numerical simulations.

[5]  arXiv:2010.11400 [pdf, ps, other]
Title: Energy-Efficient Node Deployment in Heterogeneous Two-Tier Wireless Sensor Networks with Limited Communication Range
Comments: arXiv admin note: substantial text overlap with arXiv:1901.06742
Subjects: Information Theory (cs.IT)

We study a heterogeneous two-tier wireless sensor network in which N heterogeneous access points (APs) collect sensing data from densely distributed sensors and then forward the data to M heterogeneous fusion centers (FCs). This heterogeneous node deployment problem is modeled as an optimization problem with the total power consumption of the network as its cost function. The necessary conditions of the optimal AP and FC node deployment are explored in this paper. We provide a variation of Voronoi Diagram as the optimal cell partition for this network and show that each AP should be placed between its connected FC and the geometric center of its cell partition. In addition, we propose a heterogeneous two-tier Lloyd algorithm to optimize the node deployment. Furthermore, we study the sensor deployment when the communication range is limited for sensors and APs. Simulation results show that our proposed algorithms outperform the existing clustering methods like Minimum Energy Routing, Agglomerative Clustering, Divisive Clustering, Particle Swarm Optimization, Relay Node placement in Double-tiered Wireless Sensor Networks, and Improved Relay Node Placement, on average.

[6]  arXiv:2010.11623 [pdf, ps, other]
Title: Performance Analysis and Optimization for the MAC Protocol in UAV-based IoT Network
Authors: Bin Li, Xianzhen Guo, Ruonan Zhang, Xiaojiang Du (Fellow, IEEE), Mohsen Guizani (Fellow, IEEE)
Subjects: Information Theory (cs.IT)

Unmanned aerial vehicles (UAVs) have played an important role in air-ground integration network. Especially in Internet of Things (IoT) services, UAV equipped with communication equipments is widely adopted as a mobile base station (BS) for data collection from IoT devices on the ground. In this paper, we consider an air-ground network in which the UAV flies straightly to collect information from the IoT devices in a 2-D plane based on the CSMA/CA protocol. Due to UAV's continuous mobility, the communication durations of devices in different locations with UAV are not only time-limited, but also vary from each other. To analyze the throughput performance of uplink multiple access control (MAC) protocol, we propose a new analysis model to deal with the communications heterogeneity in the network. Firstly, we divide the devices in the coverage into different clusters according to their communication durations. Then, a quitting probability indicating the probability that a device quits the UAV's coverage at each time slot is clarified. A modified three-dimensional Markov chain model adopting the quitting probability and cluster division is developed for the performance analysis. Besides, we also propose a modified CSMA/CA protocol which fully considers the heterogeneity of the access time and adaptively allocates the time resource among the devices in different clusters. Finally, the effects of retry limit, initial contention window size, the density of the devices, UAVs speed and coverage area are discussed in the simulation section.

[7]  arXiv:2010.11713 [pdf, other]
Title: Joint Power Allocation and User Association Optimization for IRS-Assisted mmWave Systems
Comments: 30 pages, 9 figures
Subjects: Information Theory (cs.IT); Systems and Control (eess.SY)

Intelligent reflecting surface (IRS) is a potential technology to build programmable wireless environment in future communication systems. In this paper, we consider an IRS-assisted multi-base station (multi-BS) multi-user millimeter wave (mmWave) downlink communication system, exploiting IRS to extend mmWave signal coverage to blind spots. Considering the impact of IRS on user association in multi-BS mmWave systems, we formulate a sum rate maximization problem by jointly optimizing passive beamforming at IRS, power allocation and user association. This leads to an intractable nonconvex problem, for which to tackle we propose a computationally affordable iterative algorithm, capitalizing on alternating optimization, sequential fractional programming (SFP) and forward-reverse auction (FRA). In particular, passive beamforming at IRS is optimized by utilizing the SFP method, power allocation is solved through means of standard convex optimization method, and user association is handled by the network optimization based FRA algorithm. Simulation results demonstrate that the proposed algorithm can achieve significant performance gains, e.g., it can provide up to 175% higher sum rate compared with the benchmark and 140% higher energy efficiency compared with amplifyand-forward relay.

[8]  arXiv:2010.11749 [pdf, ps, other]
Title: How wireless queues benefit from motion: an analysis of the continuum between zero and infinite mobility
Comments: Preliminary version appeared in WiOPT 2020
Subjects: Information Theory (cs.IT); Networking and Internet Architecture (cs.NI); Probability (math.PR)

This paper considers the time evolution of a queue that is embedded in a Poisson point process of moving wireless interferers. The queue is driven by an external arrival process and is subject to a time-varying service process that is a function of the SINR that it sees. Static configurations of interferers result in an infinite queue workload with positive probability. In contrast, a generic stability condition is established for the queue in the case where interferers possess any non-zero mobility that results in displacements that are both independent across interferers and oblivious to interferer positions. The proof leverages the mixing property of the Poisson point process. The effect of an increase in mobility on queueing metrics is also studied. Convex ordering tools are used to establish that faster moving interferers result in a queue workload that is larger for the increasing convex stochastic order. As a corollary, mean workload and mean delay improve as network mobility increases. Positive correlation between SINR level-crossing events at different time points is established and the autocorrelation function is determined. System behaviour is empirically analyzed using discrete-event simulation. The performance of various mobility models is evaluated using heavy-traffic approximations.

Cross-lists for Fri, 23 Oct 20

[9]  arXiv:2010.11533 (cross-list from cs.AI) [pdf, other]
Title: Exponential Negation of a Probability Distribution
Authors: Qinyuan Wu, Yong Deng
Comments: 6 pages, 6 figures
Subjects: Artificial Intelligence (cs.AI); Information Theory (cs.IT)

Negation operation is important in intelligent information processing. Different with existing arithmetic negation, an exponential negation is presented in this paper. The new negation can be seen as a kind of geometry negation. Some basic properties of the proposed negation is investigated, we find that the fix point is the uniform probability distribution.The negation is an entropy increase operation and all the probability distributions will converge to the uniform distribution after multiple negation iterations. The number of iterations of convergence is inversely proportional to the number of elements in the distribution. Some numerical examples are used to illustrate the efficiency of the proposed negation.

[10]  arXiv:2010.11537 (cross-list from math.ST) [pdf, ps, other]
Title: On Mean Estimation for Heteroscedastic Random Variables
Comments: 29 pages
Subjects: Statistics Theory (math.ST); Information Theory (cs.IT); Machine Learning (cs.LG)

We study the problem of estimating the common mean $\mu$ of $n$ independent symmetric random variables with different and unknown standard deviations $\sigma_1 \le \sigma_2 \le \cdots \le\sigma_n$. We show that, under some mild regularity assumptions on the distribution, there is a fully adaptive estimator $\widehat{\mu}$ such that it is invariant to permutations of the elements of the sample and satisfies that, up to logarithmic factors, with high probability, \[ |\widehat{\mu} - \mu| \lesssim \min\left\{\sigma_{m^*}, \frac{\sqrt{n}}{\sum_{i = \sqrt{n}}^n \sigma_i^{-1}} \right\}~, \] where the index $m^* \lesssim \sqrt{n}$ satisfies $m^* \approx \sqrt{\sigma_{m^*}\sum_{i = m^*}^n\sigma_i^{-1}}$.

[11]  arXiv:2010.11552 (cross-list from cs.LG) [pdf, ps, other]
Title: Nonvacuous Loss Bounds with Fast Rates for Neural Networks via Conditional Information Measures
Subjects: Machine Learning (cs.LG); Information Theory (cs.IT); Machine Learning (stat.ML)

We present a framework to derive bounds on the test loss of randomized learning algorithms for the case of bounded loss functions. This framework leads to bounds that depend on the conditional information density between the the output hypothesis and the choice of the training set, given a larger set of data samples from which the training set is formed. Furthermore, the bounds pertain to the average test loss as well as to its tail probability, both for the PAC-Bayesian and the single-draw settings. If the conditional information density is bounded uniformly in the size $n$ of the training set, our bounds decay as $1/n$, which is referred to as a fast rate. This is in contrast with the tail bounds involving conditional information measures available in the literature, which have a less benign $1/\sqrt{n}$ dependence. We demonstrate the usefulness of our tail bounds by showing that they lead to estimates of the test loss achievable with several neural network architectures trained on MNIST and Fashion-MNIST that match the state-of-the-art bounds available in the literature.

[12]  arXiv:2010.11909 (cross-list from eess.SP) [pdf, other]
Title: Contrastive Self-Supervised Learning for Wireless Power Control
Comments: Code available at this https URL
Subjects: Signal Processing (eess.SP); Information Theory (cs.IT); Machine Learning (cs.LG); Machine Learning (stat.ML)

We propose a new approach for power control in wireless networks using self-supervised learning. We partition a multi-layer perceptron that takes as input the channel matrix and outputs the power control decisions into a backbone and a head, and we show how we can use contrastive learning to pre-train the backbone so that it produces similar embeddings at its output for similar channel matrices and vice versa, where similarity is defined in an information-theoretic sense by identifying the interference links that can be optimally treated as noise. The backbone and the head are then fine-tuned using a limited number of labeled samples. Simulation results show the effectiveness of the proposed approach, demonstrating significant gains over pure supervised learning methods in both sum-throughput and sample efficiency.

[13]  arXiv:2010.11914 (cross-list from math.ST) [pdf, ps, other]
Title: Gaussoids are two-antecedental approximations of Gaussian conditional independence structures
Authors: Tobias Boege
Comments: 17 pages
Subjects: Statistics Theory (math.ST); Information Theory (cs.IT)

The gaussoid axioms are conditional independence inference rules which characterize regular Gaussian CI structures over a three-element ground set. It is known that no finite set of inference rules completely describes regular Gaussian CI as the ground set grows. In this article we show that the gaussoid axioms logically imply every inference rule of at most two antecedents which is valid for regular Gaussians over any ground set. The proof is accomplished by exhibiting for each inclusion-minimal gaussoid extension of at most two CI statements a regular Gaussian realization. Moreover we prove that all those gaussoids have rational positive-definite realizations inside every $\varepsilon$-ball around the identity matrix. For the proof we introduce the concept of algebraic Gaussians over arbitrary fields and of positive Gaussians over ordered fields and obtain the same two-antecedental completeness of the gaussoid axioms for algebraic and positive Gaussians over all fields of characteristic zero as a byproduct.

[14]  arXiv:2010.11935 (cross-list from cs.DC) [pdf, other]
Title: Coded Data Rebalancing for Decentralized Distributed Databases
Subjects: Distributed, Parallel, and Cluster Computing (cs.DC); Information Theory (cs.IT)

The performance of replication-based distributed databases is affected due to non-uniform storage across storage nodes (also called \textit{data skew}) and reduction in the replication factor during operation, particularly due to node additions or removals. Data rebalancing refers to the communication involved between the nodes in correcting this data skew, while maintaining the replication factor. For carefully designed distributed databases, transmitting coded symbols during the rebalancing phase has been recently shown to reduce the communication load of rebalancing. In this work, we look at balanced distributed databases with \textit{random placement}, in which each data segment is stored in a random subset of $r$ nodes in the system, where $r$ refers to the replication factor of the distributed database. We call these as decentralized databases. For a natural class of such decentralized databases, we propose rebalancing schemes for correcting data skew and reinstating the replication factor arising due to a single node addition or removal. We give converse arguments which show that our proposed rebalancing schemes are optimal asymptotically in the size of the file.

Replacements for Fri, 23 Oct 20

[15]  arXiv:1812.07026 (replaced) [pdf, ps, other]
Title: State Leakage and Coordination with Causal State Knowledge at the Encoder
Comments: preliminary draft
Subjects: Information Theory (cs.IT)
[16]  arXiv:2001.10290 (replaced) [pdf, other]
Title: Discrete Signal Processing with Set Functions
Comments: 16 pages, submitted for publication
Subjects: Information Theory (cs.IT); Machine Learning (cs.LG); Signal Processing (eess.SP)
[17]  arXiv:2002.05100 (replaced) [src]
Title: Fundamental Limits of Biometric Identification Systems with Strong Secrecy
Comments: Some critical errors are found in the proofs of the main results, so we want to withdraw this paper. Thank you
Subjects: Information Theory (cs.IT)
[18]  arXiv:2005.08545 (replaced) [pdf, ps, other]
Title: Joint Index Coding and Incentive Design for Selfish Clients
Comments: 37 pages (single column), submitted for possible journal publication
Subjects: Information Theory (cs.IT); Networking and Internet Architecture (cs.NI)
[19]  arXiv:2006.11727 (replaced) [pdf, other]
Title: Affine symmetries and neural network identifiability
Comments: 59 pages, 9 figures
Subjects: Information Theory (cs.IT); Machine Learning (cs.LG); Machine Learning (stat.ML)
[20]  arXiv:2006.12147 (replaced) [pdf, other]
Title: Optimization of NB QC-LDPC Block Codes and Their Performance Analysis
Comments: 26 pages
Subjects: Information Theory (cs.IT)
[21]  arXiv:1807.07648 (replaced) [pdf, ps, other]
Title: An improved uncertainty principle for functions with symmetry
Comments: 29 pages
Subjects: Classical Analysis and ODEs (math.CA); Information Theory (cs.IT); Number Theory (math.NT)
[22]  arXiv:2005.01580 (replaced) [pdf, ps, other]
Title: How Many Modes Can a Mixture of Gaussians with Uniformly Bounded Means Have?
Comments: 11 pages, 1 figure; this version is currently under review at Information and Inference: A Journal of the IMA
Subjects: Statistics Theory (math.ST); Information Theory (cs.IT)
[23]  arXiv:2007.11078 (replaced) [pdf, other]
Title: The Complete Lasso Tradeoff Diagram
Subjects: Statistics Theory (math.ST); Information Theory (cs.IT)
[24]  arXiv:2010.09484 (replaced) [pdf, other]
Title: Information-Theoretic Bounds on Transfer Generalization Gap Based on Jensen-Shannon Divergence
Comments: Submitted for conference publication
Subjects: Machine Learning (cs.LG); Information Theory (cs.IT); Signal Processing (eess.SP)
[ total of 24 entries: 1-24 ]
[ showing up to 2000 entries per page: fewer | more ]

Disable MathJax (What is MathJax?)

Links to: arXiv, form interface, find, cs, recent, 2010, contact, help  (Access key information)