Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Robust model training and generalisation with Studentising flows
(Submitted on 11 Jun 2020 (v1), last revised 11 Jul 2020 (this version, v2))
Abstract: Normalising flows are tractable probabilistic models that leverage the power of deep learning to describe a wide parametric family of distributions, all while remaining trainable using maximum likelihood. We discuss how these methods can be further improved based on insights from robust (in particular, resistant) statistics. Specifically, we propose to endow flow-based models with fat-tailed latent distributions such as multivariate Student's $t$, as a simple drop-in replacement for the Gaussian distribution used by conventional normalising flows. While robustness brings many advantages, this paper explores two of them: 1) We describe how using fatter-tailed base distributions can give benefits similar to gradient clipping, but without compromising the asymptotic consistency of the method. 2) We also discuss how robust ideas lead to models with reduced generalisation gap and improved held-out data likelihood. Experiments on several different datasets confirm the efficacy of the proposed approach in both regards.
Submission history
From: Gustav Eje Henter [view email][v1] Thu, 11 Jun 2020 16:47:01 GMT (450kb,D)
[v2] Sat, 11 Jul 2020 12:50:13 GMT (528kb,D)
Link back to: arXiv, form interface, contact.