We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

math.NA

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Numerical Analysis

Title: Swarm-Based Gradient Descent Method for Non-Convex Optimization

Abstract: We introduce a new Swarm-Based Gradient Descent (SBGD) method for non-convex optimization. The swarm consists of agents, each is identified with a position, $\boldsymbol{x}$, and mass, $m$. The key to their dynamics is communication: masses are being transferred from agents at high ground to low(-est) ground. At the same time, agents change positions with step size, $h=h(\boldsymbol{x},m)$, adjusted to their relative mass: heavier agents proceed with small time-steps in the direction of local gradient, while lighter agents take larger time-steps based on a backtracking protocol. Accordingly, the crowd of agents is dynamically divided between `heavier' leaders, expected to approach local minima, and `lighter' explorers. With their large-step protocol, explorers are expected to encounter improved position for the swarm; if they do, then they assume the role of `heavy' swarm leaders and so on. Convergence analysis and numerical simulations in one-, two-, and 20-dimensional benchmarks demonstrate the effectiveness of SBGD as a global optimizer.
Subjects: Numerical Analysis (math.NA); Optimization and Control (math.OC)
MSC classes: 90C26, 65K10, 92D25
Cite as: arXiv:2211.17157 [math.NA]
  (or arXiv:2211.17157v1 [math.NA] for this version)

Submission history

From: Eitan Tadmor [view email]
[v1] Wed, 30 Nov 2022 16:47:15 GMT (777kb,D)

Link back to: arXiv, form interface, contact.