We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

math.OC

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Optimization and Control

Title: Stochastic Langevin Differential Inclusions with Applications to Machine Learning

Abstract: Stochastic differential equations of Langevin-diffusion form have received significant recent, thanks to their foundational role in both Bayesian sampling algorithms and optimization in machine learning. In the latter, they serve as a conceptual model of the stochastic gradient flow in training over-parametrized models. However, the literature typically assumes smoothness of the potential, whose gradient is the drift term. Nevertheless, there are many problems, for which the potential function is not continuously differentiable, and hence the drift is not Lipschitz-continuous everywhere. This is exemplified by robust losses and Rectified Linear Units in regression problems. In this paper, we show some foundational results regarding the flow and asymptotic properties of Langevin-type Stochastic Differential Inclusions under assumptions appropriate to the machine-learning settings. In particular, we show strong existence of the solution, as well as asymptotic minimization of the canonical Free Energy Functional.
Comments: 22 pages, 3 figures
Subjects: Optimization and Control (math.OC); Machine Learning (cs.LG); Numerical Analysis (math.NA)
Cite as: arXiv:2206.11533 [math.OC]
  (or arXiv:2206.11533v1 [math.OC] for this version)

Submission history

From: Jakub Marecek [view email]
[v1] Thu, 23 Jun 2022 08:29:17 GMT (113kb,D)

Link back to: arXiv, form interface, contact.