References & Citations
Statistics > Methodology
Title: The Difficult Task of Distribution Generalization in Nonlinear Models
(Submitted on 12 Jun 2020 (this version), latest version 17 Aug 2021 (v3))
Abstract: We consider the problem of predicting a response from a set of covariates when the test distribution differs from the training distribution. Here, we consider robustness against distributions that emerge as intervention distributions. Causal models that regress the response variable on all of its causal parents have been suggested for the above task since they remain valid under arbitrary interventions on any subset of covariates. However, in linear models, for a set of interventions with bounded strength, alternative approaches have been shown to be minimax prediction optimal. In this work, we analyze minimax solutions in nonlinear models for both direct and indirect interventions on the covariates. We prove that the causal function is minimax optimal for a large class of interventions. We introduce the notion of distribution generalization, which is motivated by the fact that, in practice, minimax solutions need to be identified from observational data. We prove sufficient conditions for distribution generalization and present corresponding impossibility results. To illustrate the above findings, we propose a practical method, called NILE, that achieves distribution generalization in a nonlinear instrumental variable setting with linear extrapolation. We prove consistency, present empirical results and provide code.
Submission history
From: Rune Christiansen [view email][v1] Fri, 12 Jun 2020 19:24:02 GMT (915kb,D)
[v2] Fri, 4 Sep 2020 20:00:08 GMT (918kb,D)
[v3] Tue, 17 Aug 2021 19:00:01 GMT (1402kb,D)
Link back to: arXiv, form interface, contact.