We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.LG

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: Bayesian Optimization for auto-tuning GPU kernels

Abstract: Finding optimal parameter configurations for tunable GPU kernels is a non-trivial exercise for large search spaces, even when automated. This poses an optimization task on a non-convex search space, using an expensive to evaluate function with unknown derivative. These characteristics make a good candidate for Bayesian Optimization, which has not been applied to this problem before. However, the application of Bayesian Optimization to this problem is challenging. We demonstrate how to deal with the rough, discrete, constrained search spaces, containing invalid configurations. We introduce a novel contextual variance exploration factor, as well as new acquisition functions with improved scalability, combined with an informed acquisition function selection mechanism. By comparing the performance of our Bayesian Optimization implementation on various test cases to the existing search strategies in Kernel Tuner, as well as other Bayesian Optimization implementations, we demonstrate that our search strategies generalize well and consistently outperform other search strategies by a wide margin.
Comments: In context of the 2021 International Workshop on Performance Modeling, Benchmarking and Simulation of High Performance Computer Systems (PMBS) at SuperComputing 2021. Pending publication in IEEE Computer Society Technical Consortium on High Performance Computing (TCHPC)
Subjects: Machine Learning (cs.LG); Distributed, Parallel, and Cluster Computing (cs.DC); Performance (cs.PF); Optimization and Control (math.OC)
ACM classes: D.1.3; C.4; G.3; I.6.3
Cite as: arXiv:2111.14991 [cs.LG]
  (or arXiv:2111.14991v1 [cs.LG] for this version)

Submission history

From: Floris-Jan Willemsen [view email]
[v1] Fri, 26 Nov 2021 11:26:26 GMT (3969kb,D)

Link back to: arXiv, form interface, contact.