We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Machine Learning

Title: Meta-strategy for Learning Tuning Parameters with Guarantees

Abstract: Online learning methods, like the online gradient algorithm (OGA) and exponentially weighted aggregation (EWA), often depend on tuning parameters that are difficult to set in practice. We consider an online meta-learning scenario, and we propose a meta-strategy to learn these parameters from past tasks. Our strategy is based on the minimization of a regret bound. It allows to learn the initialization and the step size in OGA with guarantees. It also allows to learn the prior or the learning rate in EWA. We provide a regret analysis of the strategy. It allows to identify settings where meta-learning indeed improves on learning each task in isolation.
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Statistics Theory (math.ST); Computation (stat.CO)
Journal reference: Entropy, 2021, vol. 23, no. 10, 1257
DOI: 10.3390/e23101257
Cite as: arXiv:2102.02504 [stat.ML]
  (or arXiv:2102.02504v3 [stat.ML] for this version)

Submission history

From: Pierre Alquier [view email]
[v1] Thu, 4 Feb 2021 09:32:22 GMT (227kb,D)
[v2] Wed, 2 Jun 2021 09:03:58 GMT (228kb,D)
[v3] Fri, 6 Aug 2021 10:52:15 GMT (268kb,D)

Link back to: arXiv, form interface, contact.