We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.LG

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: Highly-Smooth Zero-th Order Online Optimization Vianney Perchet

Authors: Francis Bach (SIERRA, LIENS), Vianney Perchet (CREST)
Abstract: The minimization of convex functions which are only available through partial and noisy information is a key methodological problem in many disciplines. In this paper we consider convex optimization with noisy zero-th order information, that is noisy function evaluations at any desired point. We focus on problems with high degrees of smoothness, such as logistic regression. We show that as opposed to gradient-based algorithms, high-order smoothness may be used to improve estimation rates, with a precise dependence of our upper-bounds on the degree of smoothness. In particular, we show that for infinitely differentiable functions, we recover the same dependence on sample size as gradient-based algorithms, with an extra dimension-dependent factor. This is done for both convex and strongly-convex functions, with finite horizon and anytime algorithms. Finally, we also recover similar results in the online optimization setting.
Comments: Conference on Learning Theory (COLT), Jun 2016, New York, United States. 2016
Subjects: Machine Learning (cs.LG); Optimization and Control (math.OC)
Cite as: arXiv:1605.08165 [cs.LG]
  (or arXiv:1605.08165v1 [cs.LG] for this version)

Submission history

From: Francis Bach [view email]
[v1] Thu, 26 May 2016 06:54:55 GMT (33kb)

Link back to: arXiv, form interface, contact.