We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.IT

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Information Theory

Title: Universal Weak Variable-Length Source Coding on Countable Infinite Alphabets

Abstract: Motivated from the fact that universal source coding on countably infinite alphabets is not feasible, this work introduces the notion of almost lossless source coding. Analog to the weak variable-length source coding problem studied by Han in 2000, almost lossless source coding aims at relaxing the lossless block-wise assumption to allow an average per-letter distortion that vanishes asymptotically as the block-length goes to infinity. In this setup, we show on one hand that Shannon entropy characterizes the minimum achievable rate (similarly to the case of discrete sources) while on the other that almost lossless universal source coding becomes feasible for the family of finite-entropy stationary memoryless sources with countably infinite alphabets. Furthermore, we study a stronger notion of almost lossless universality that demands uniform convergence of the average per-letter distortion to zero, where we establish a necessary and sufficient condition for the so-called family of envelope distributions to achieve it. Remarkably, this condition is the same necessary and sufficient condition needed for the existence of a strongly minimax (lossless) universal source code for the family of envelope distributions. Finally, we show that an almost lossless coding scheme offers faster rate of convergence for the (minimax) redundancy compared to the well-known information radius developed for the lossless case at the expense of tolerating a non-zero distortion that vanishes to zero as the block-length grows. This shows that even when lossless universality is feasible, an almost lossless scheme can offer different regimes on the rates of convergence of the (worst case) redundancy versus the (worst case) distortion.
Comments: This paper has been submitted for review to the IEEE Trans. on Information Theory on August 2017. 37 pages. The material in this paper was partially published in ISIT2016 [1] and ISIT2017 [2], International Symposium on Information Theory (ISIT)
Subjects: Information Theory (cs.IT)
Cite as: arXiv:1708.08103 [cs.IT]
  (or arXiv:1708.08103v1 [cs.IT] for this version)

Submission history

From: Jorge Silva [view email]
[v1] Sun, 27 Aug 2017 16:25:52 GMT (113kb)
[v2] Wed, 22 Sep 2021 17:05:03 GMT (201kb,D)
[v3] Fri, 26 Nov 2021 21:55:46 GMT (201kb,D)

Link back to: arXiv, form interface, contact.