We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

math.NA

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Numerical Analysis

Title: Leveraging Memory Effects and Gradient Information in Consensus-Based Optimization: On Global Convergence in Mean-Field Law

Abstract: In this paper we study consensus-based optimization (CBO), a versatile, flexibel and customizable optimization method suitable for performing nonconvex and nonsmooth global optimizations in high dimensions. CBO is a multi-particle metaheuristic, which is effective in various applications and at the same time amenable to theoretical analysis thanks to its minimalistic design. The underlying dynamics, however, is flexible enough to incorporate different mechanisms widely used in evolutionary computation and machine learning, as we show by analyzing a variant of CBO which makes use of memory effects and gradient information. We rigorously prove that this dynamics converges to a global minimizer of the objective function in mean-field law for a vast class of functions under minimal assumptions on the initialization of the method. The proof in particular reveals how to leverage further, in some applications advantageous, forces in the dynamics without loosing provable global convergence. To demonstrate the benefit of the herein investigated memory effects and gradient information in certain applications, we present numerical evidence for the superiority of this CBO variant in applications such as machine learning and compressed sensing, which en passant widen the scope of applications of CBO.
Comments: 31 pages, 6 figures
Subjects: Numerical Analysis (math.NA); Neural and Evolutionary Computing (cs.NE); Analysis of PDEs (math.AP); Optimization and Control (math.OC)
MSC classes: 65K10, 90C26, 90C56, 35Q90, 35Q84
Cite as: arXiv:2211.12184 [math.NA]
  (or arXiv:2211.12184v1 [math.NA] for this version)

Submission history

From: Konstantin Riedl [view email]
[v1] Tue, 22 Nov 2022 11:22:11 GMT (1620kb,D)

Link back to: arXiv, form interface, contact.