Current browse context:
cs.IT
Change to browse by:
References & Citations
Computer Science > Information Theory
Title: Principal Bit Analysis: Autoencoding with Schur-Concave Loss
(Submitted on 5 Jun 2021 (v1), last revised 8 Jun 2021 (this version, v2))
Abstract: We consider a linear autoencoder in which the latent variables are quantized, or corrupted by noise, and the constraint is Schur-concave in the set of latent variances. Although finding the optimal encoder/decoder pair for this setup is a nonconvex optimization problem, we show that decomposing the source into its principal components is optimal. If the constraint is strictly Schur-concave and the empirical covariance matrix has only simple eigenvalues, then any optimal encoder/decoder must decompose the source in this way. As one application, we consider a strictly Schur-concave constraint that estimates the number of bits needed to represent the latent variables under fixed-rate encoding, a setup that we call \emph{Principal Bit Analysis (PBA)}. This yields a practical, general-purpose, fixed-rate compressor that outperforms existing algorithms. As a second application, we show that a prototypical autoencoder-based variable-rate compressor is guaranteed to decompose the source into its principal components.
Submission history
From: Sourbh Bhadane [view email][v1] Sat, 5 Jun 2021 04:45:30 GMT (1354kb,D)
[v2] Tue, 8 Jun 2021 16:16:13 GMT (3844kb,D)
Link back to: arXiv, form interface, contact.