References & Citations
Computer Science > Information Theory
Title: Mathematics of Sparsity and Entropy: Axioms, Core Functions and Sparse Recovery
(Submitted on 21 Jan 2015 (v1), last revised 16 Dec 2015 (this version, v3))
Abstract: Sparsity and entropy are pillar notions of modern theories in signal processing and information theory. However, there is no clear consensus among scientists on the characterization of these notions. Previous efforts have contributed to understand individually sparsity or entropy from specific research interests. This paper proposes a mathematical formalism, a joint axiomatic characterization, which contributes to comprehend (the beauty of) sparsity and entropy. The paper gathers and introduces inherent and first principles criteria as axioms and attributes that jointly characterize sparsity and entropy. The proposed set of axioms is constructive and allows to derive simple or \emph{core functions} and further generalizations. Core sparsity generalizes the Hoyer measure, Gini index and $pq$-means. Core entropy generalizes the R\'{e}nyi entropy and Tsallis entropy, both of which generalize Shannon entropy. Finally, core functions are successfully applied to compressed sensing and to minimum entropy given sample moments. More importantly, the (simplest) core sparsity adds theoretical support to the $\ell_1$-minimization approach in compressed sensing.
Submission history
From: Giancarlo Pastor [view email][v1] Wed, 21 Jan 2015 11:04:56 GMT (258kb,D)
[v2] Thu, 22 Jan 2015 08:18:08 GMT (262kb,D)
[v3] Wed, 16 Dec 2015 21:04:00 GMT (0kb,I)
Link back to: arXiv, form interface, contact.