Current browse context:
stat.ML
Change to browse by:
References & Citations
Statistics > Machine Learning
Title: Optimum-statistical Collaboration Towards General and Efficient Black-box Optimization
(Submitted on 17 Jun 2021 (v1), last revised 31 May 2023 (this version, v5))
Abstract: In this paper, we make the key delineation on the roles of resolution and statistical uncertainty in hierarchical bandits-based black-box optimization algorithms, guiding a more general analysis and a more efficient algorithm design. We introduce the \textit{optimum-statistical collaboration}, an algorithm framework of managing the interaction between optimization error flux and statistical error flux evolving in the optimization process. We provide a general analysis of this framework without specifying the forms of statistical error and uncertainty quantifier. Our framework and its analysis, due to their generality, can be applied to a large family of functions and partitions that satisfy different local smoothness assumptions and have different numbers of local optimums, which is much richer than the class of functions studied in prior works. Our framework also inspires us to propose a better measure of the statistical uncertainty and consequently a variance-adaptive algorithm \texttt{VHCT}. In theory, we prove the algorithm enjoys rate-optimal regret bounds under different local smoothness assumptions; in experiments, we show the algorithm outperforms prior efforts in different settings.
Submission history
From: Wenjie Li [view email][v1] Thu, 17 Jun 2021 02:37:39 GMT (34163kb,D)
[v2] Sun, 17 Oct 2021 01:22:44 GMT (2334kb,D)
[v3] Tue, 8 Mar 2022 02:53:57 GMT (2333kb,D)
[v4] Tue, 7 Mar 2023 18:23:05 GMT (1613kb,D)
[v5] Wed, 31 May 2023 05:07:00 GMT (1615kb,D)
Link back to: arXiv, form interface, contact.