We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ML

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Machine Learning

Title: Does Invariant Risk Minimization Capture Invariance?

Abstract: We show that the Invariant Risk Minimization (IRM) formulation of Arjovsky et al. (2019) can fail to capture "natural" invariances, at least when used in its practical "linear" form, and even on very simple problems which directly follow the motivating examples for IRM. This can lead to worse generalization on new environments, even when compared to unconstrained ERM. The issue stems from a significant gap between the linear variant (as in their concrete method IRMv1) and the full non-linear IRM formulation. Additionally, even when capturing the "right" invariances, we show that it is possible for IRM to learn a sub-optimal predictor, due to the loss function not being invariant across environments. The issues arise even when measuring invariance on the population distributions, but are exacerbated by the fact that IRM is extremely fragile to sampling.
Comments: Code is available in the arXiv ancillary files, linked from this page
Subjects: Machine Learning (stat.ML); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
Cite as: arXiv:2101.01134 [stat.ML]
  (or arXiv:2101.01134v2 [stat.ML] for this version)

Submission history

From: Danica J. Sutherland [view email]
[v1] Mon, 4 Jan 2021 18:02:45 GMT (2644kb,AD)
[v2] Fri, 26 Feb 2021 23:21:48 GMT (4785kb,AD)

Link back to: arXiv, form interface, contact.