We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

math.OC

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Optimization and Control

Title: Uncertainty Quantification for Gradient and Accelerated Gradient Descent Methods on Strongly Convex Functions

Abstract: We consider the problem of minimizing a strongly convex function that depends on an uncertain parameter $\theta$. The uncertainty in the objective function means that the optimum, $x^*(\theta)$, is also a function of $\theta$. We propose an efficient method to compute $x^*(\theta)$ and its statistics. We use a chaos expansion of $x^*(\theta)$ along a truncated basis and study first-order methods that compute the optimal coefficients. We establish the convergence rate of the method as the number of basis functions, and hence the dimensionality of the optimization problem is increased. We give the first non-asymptotic rates for the gradient descent and the accelerated gradient descent methods. Our analysis exploits convexity and does not rely on a diminishing step-size strategy. As a result, it is much faster than the state-of-the-art both in theory and in our preliminary numerical experiments. A surprising side-effect of our analysis is that the proposed method also acts as a variance reduction technique to the problem of estimating $x^*(\theta)$.
Comments: Minor presentation edits only. Submitted to SIAM UQ Journal
Subjects: Optimization and Control (math.OC)
MSC classes: 65K10, 41A45
ACM classes: G.1.2; G.1.6
Cite as: arXiv:2111.02836 [math.OC]
  (or arXiv:2111.02836v2 [math.OC] for this version)

Submission history

From: Conor McMeel [view email]
[v1] Thu, 4 Nov 2021 12:56:20 GMT (101kb)
[v2] Wed, 1 Dec 2021 01:37:03 GMT (102kb)

Link back to: arXiv, form interface, contact.