We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:


Current browse context:


Change to browse by:


References & Citations

DBLP - CS Bibliography


(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: Deep Ranking Ensembles for Hyperparameter Optimization

Abstract: Automatically optimizing the hyperparameters of Machine Learning algorithms is one of the primary open questions in AI. Existing work in Hyperparameter Optimization (HPO) trains surrogate models for approximating the response surface of hyperparameters as a regression task. In contrast, we hypothesize that the optimal strategy for training surrogates is to preserve the ranks of the performances of hyperparameter configurations as a Learning to Rank problem. As a result, we present a novel method that meta-learns neural network surrogates optimized for ranking the configurations' performances while modeling their uncertainty via ensembling. In a large-scale experimental protocol comprising 12 baselines, 16 HPO search spaces and 86 datasets/tasks, we demonstrate that our method achieves new state-of-the-art results in HPO.
Comments: Published in ICLR 2023
Subjects: Machine Learning (cs.LG)
Cite as: arXiv:2303.15212 [cs.LG]
  (or arXiv:2303.15212v2 [cs.LG] for this version)

Submission history

From: Sebastian Pineda Arango [view email]
[v1] Mon, 27 Mar 2023 13:52:40 GMT (12475kb,D)
[v2] Sun, 21 May 2023 13:31:47 GMT (12481kb,D)

Link back to: arXiv, form interface, contact.