Current browse context:
stat.ML
Change to browse by:
References & Citations
Statistics > Machine Learning
Title: Near-optimal Oracle-efficient Algorithms for Stationary and Non-Stationary Stochastic Linear Bandits
(Submitted on 11 Dec 2019 (this version), latest version 11 Oct 2021 (v5))
Abstract: We investigate the design of two algorithms that enjoy not only computational efficiency induced by Hannan's perturbation approach, but also minimax-optimal regret bounds in linear bandit problems where the learner has access to an offline optimization oracle. We present an algorithm called Follow-The-Gaussian-Perturbed Leader (FTGPL) for stationary linear bandit where each action is associated with a $d$-dimensional feature vector, and prove that FTGPL (1) achieves the minimax-optimal $\tilde{\mathcal{O}}(d\sqrt{T})$ regret, (2) matches the empirical performance of Linear Thompson Sampling, and (3) can be efficiently implemented even in the case of infinite actions, thus achieving the best of three worlds. Furthermore, it firmly solves an open problem raised in \citet{abeille2017linear}, which perturbation achieves minimax-optimality in Linear Thompson Sampling. The weighted variant with exponential discounting, Discounted Follow-The-Gaussian-Perturbed Leader (D-FTGPL) is proposed to gracefully adjust to non-stationary environment where unknown parameter is time-varying within total variation $B_T$. It asymptotically achieves optimal dynamic regret $\tilde{\mathcal{O}}( d ^{2/3}B_T^{1/3} T^{2/3})$ and is oracle-efficient due to access to an offline optimization oracle induced by Gaussian perturbation.
Submission history
From: Baekjin Kim [view email][v1] Wed, 11 Dec 2019 23:34:12 GMT (49kb)
[v2] Wed, 15 Jan 2020 00:33:57 GMT (49kb)
[v3] Thu, 20 Feb 2020 18:40:14 GMT (1137kb,D)
[v4] Fri, 12 Jun 2020 17:04:55 GMT (1159kb,D)
[v5] Mon, 11 Oct 2021 20:23:25 GMT (2729kb,D)
Link back to: arXiv, form interface, contact.