We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ML

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Machine Learning

Title: Diffusion Schrödinger Bridge with Applications to Score-Based Generative Modeling

Abstract: Progressively applying Gaussian noise transforms complex data distributions to approximately Gaussian. Reversing this dynamic defines a generative model. When the forward noising process is given by a Stochastic Differential Equation (SDE), Song et al. (2021) demonstrate how the time inhomogeneous drift of the associated reverse-time SDE may be estimated using score-matching. A limitation of this approach is that the forward-time SDE must be run for a sufficiently long time for the final distribution to be approximately Gaussian. In contrast, solving the Schr\"odinger Bridge problem (SB), i.e. an entropy-regularized optimal transport problem on path spaces, yields diffusions which generate samples from the data distribution in finite time. We present Diffusion SB (DSB), an original approximation of the Iterative Proportional Fitting (IPF) procedure to solve the SB problem, and provide theoretical analysis along with generative modeling experiments. The first DSB iteration recovers the methodology proposed by Song et al. (2021), with the flexibility of using shorter time intervals, as subsequent DSB iterations reduce the discrepancy between the final-time marginal of the forward (resp. backward) SDE with respect to the prior (resp. data) distribution. Beyond generative modeling, DSB offers a widely applicable computational optimal transport tool as the continuous state-space analogue of the popular Sinkhorn algorithm (Cuturi, 2013).
Comments: 58 pages, 18 figures (correction of Proposition 5)
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Probability (math.PR)
Cite as: arXiv:2106.01357 [stat.ML]
  (or arXiv:2106.01357v4 [stat.ML] for this version)

Submission history

From: Valentin De Bortoli [view email]
[v1] Tue, 1 Jun 2021 17:34:27 GMT (10112kb,D)
[v2] Wed, 16 Jun 2021 13:27:03 GMT (10387kb,D)
[v3] Tue, 9 Nov 2021 17:24:46 GMT (9738kb,D)
[v4] Mon, 6 Dec 2021 21:06:08 GMT (9738kb,D)

Link back to: arXiv, form interface, contact.