We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Machine Learning

Title: Blindness of score-based methods to isolated components and mixing proportions

Abstract: Statistical tasks such as density estimation and approximate Bayesian inference often involve densities with unknown normalising constants. Score-based methods, including score matching, are popular techniques as they are free of normalising constants. Although these methods enjoy theoretical guarantees, a little-known fact is that they suffer from practical failure modes when the unnormalised distribution of interest has isolated components -- they cannot discover isolated components or identify the correct mixing proportions between components. We demonstrate these findings using simple distributions and present heuristic attempts to address these issues. We hope to bring the attention of theoreticians and practitioners to these issues when developing new algorithms and applications.
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)
Cite as: arXiv:2008.10087 [stat.ML]
  (or arXiv:2008.10087v2 [stat.ML] for this version)

Submission history

From: Li Wenliang [view email]
[v1] Sun, 23 Aug 2020 18:24:42 GMT (2950kb,D)
[v2] Tue, 21 Sep 2021 22:10:24 GMT (9432kb,D)

Link back to: arXiv, form interface, contact.