We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ME

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Machine Learning

Title: Desiderata for Representation Learning: A Causal Perspective

Abstract: Representation learning constructs low-dimensional representations to summarize essential features of high-dimensional data. This learning problem is often approached by describing various desiderata associated with learned representations; e.g., that they be non-spurious, efficient, or disentangled. It can be challenging, however, to turn these intuitive desiderata into formal criteria that can be measured and enhanced based on observed data. In this paper, we take a causal perspective on representation learning, formalizing non-spuriousness and efficiency (in supervised representation learning) and disentanglement (in unsupervised representation learning) using counterfactual quantities and observable consequences of causal assertions. This yields computable metrics that can be used to assess the degree to which representations satisfy the desiderata of interest and learn non-spurious and disentangled representations from single observational datasets.
Comments: 68 pages
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Methodology (stat.ME)
Cite as: arXiv:2109.03795 [stat.ML]
  (or arXiv:2109.03795v2 [stat.ML] for this version)

Submission history

From: Yixin Wang [view email]
[v1] Wed, 8 Sep 2021 17:33:54 GMT (5374kb,D)
[v2] Thu, 10 Feb 2022 23:00:52 GMT (5402kb,D)

Link back to: arXiv, form interface, contact.