We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

physics

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Optimization and Control

Title: Generalization to the Natural Gradient Descent

Abstract: Optimization problem, which is aimed at finding the global minimal value of a given cost function, is one of the central problem in science and engineering. Various numerical methods have been proposed to solve this problem, among which the Gradient Descent (GD) method is the most popular one due to its simplicity and efficiency. However, the GD method suffers from two main issues: the local minima and the slow convergence especially near the minima point. The Natural Gradient Descent(NGD), which has been proved as one of the most powerful method for various optimization problems in machine learning, tensor network, variational quantum algorithms and so on, supplies an efficient way to accelerate the convergence. Here, we give a unified method to extend the NGD method to a more general situation which keeps the fast convergence by looking for a more suitable metric through introducing a 'proper' reference Riemannian manifold. Our method generalizes the NDG, and may give more insight of the optimization methods.
Subjects: Optimization and Control (math.OC); Computational Physics (physics.comp-ph)
Cite as: arXiv:2210.02764 [math.OC]
  (or arXiv:2210.02764v1 [math.OC] for this version)

Submission history

From: Shaojun Dong [view email]
[v1] Thu, 6 Oct 2022 09:04:14 GMT (288kb,D)

Link back to: arXiv, form interface, contact.