We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ML

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Machine Learning

Title: Indeterminacy in Latent Variable Models: Characterization and Strong Identifiability

Abstract: Most modern latent variable and probabilistic generative models, such as the variational autoencoder (VAE), have certain indeterminacies that are unresolvable even with an infinite amount of data. Recent applications of such models have indicated the need for strongly identifiable models, in which an observation corresponds to a unique latent code. Progress has been made towards reducing model indeterminacies while maintaining flexibility, most notably by the iVAE (arXiv:1907.04809 [stat.ML]), which excludes many -- but not all -- indeterminacies. We construct a full theoretical framework for analyzing the indeterminacies of latent variable models, and characterize them precisely in terms of properties of the generator functions and the latent variable prior distributions. To illustrate, we apply the framework to better understand the structure of recent identifiability results. We then investigate how we might specify strongly identifiable latent variable models, and construct two such classes of models. One is a straightforward modification of iVAE; the other uses ideas from optimal transport and leads to novel models and connections to recent work.
Comments: 40 pages, 2 figures. Version 2 fixes formatting and equation numbering
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)
Cite as: arXiv:2206.00801 [stat.ML]
  (or arXiv:2206.00801v2 [stat.ML] for this version)

Submission history

From: Quanhan Xi [view email]
[v1] Thu, 2 Jun 2022 00:01:27 GMT (81kb)
[v2] Mon, 20 Jun 2022 06:54:21 GMT (84kb)

Link back to: arXiv, form interface, contact.