Current browse context:
stat.ML
Change to browse by:
References & Citations
Statistics > Machine Learning
Title: Relaxing Bijectivity Constraints with Continuously Indexed Normalising Flows
(Submitted on 30 Sep 2019 (v1), last revised 23 Apr 2021 (this version, v5))
Abstract: We show that normalising flows become pathological when used to model targets whose supports have complicated topologies. In this scenario, we prove that a flow must become arbitrarily numerically noninvertible in order to approximate the target closely. This result has implications for all flow-based models, and especially Residual Flows (ResFlows), which explicitly control the Lipschitz constant of the bijection used. To address this, we propose Continuously Indexed Flows (CIFs), which replace the single bijection used by normalising flows with a continuously indexed family of bijections, and which can intuitively "clean up" mass that would otherwise be misplaced by a single bijection. We show theoretically that CIFs are not subject to the same topological limitations as normalising flows, and obtain better empirical performance on a variety of models and benchmarks.
Submission history
From: Rob Cornish [view email][v1] Mon, 30 Sep 2019 16:51:48 GMT (4733kb,D)
[v2] Thu, 20 Feb 2020 18:25:10 GMT (5081kb,D)
[v3] Thu, 16 Jul 2020 15:20:23 GMT (6595kb,D)
[v4] Fri, 14 Aug 2020 16:54:30 GMT (6595kb,D)
[v5] Fri, 23 Apr 2021 17:48:10 GMT (6595kb,D)
Link back to: arXiv, form interface, contact.