We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:


Current browse context:


Change to browse by:

References & Citations


(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Machine Learning

Title: Randomized Exploration for Non-Stationary Stochastic Linear Bandits

Abstract: We investigate two perturbation approaches to overcome conservatism that optimism based algorithms chronically suffer from in practice. The first approach replaces optimism with a simple randomization when using confidence sets. The second one adds random perturbations to its current estimate before maximizing the expected reward. For non-stationary linear bandits, where each action is associated with a $d$-dimensional feature and the unknown parameter is time-varying with total variation $B_T$, we propose two randomized algorithms, Discounted Randomized LinUCB (D-RandLinUCB) and Discounted Linear Thompson Sampling (D-LinTS) via the two perturbation approaches. We highlight the statistical optimality versus computational efficiency trade-off between them in that the former asymptotically achieves the optimal dynamic regret $\tilde{\mathcal{O}}(d ^{2/3}B_T^{1/3} T^{2/3})$, but the latter is oracle-efficient with an extra logarithmic factor in the number of arms compared to minimax-optimal dynamic regret. In a simulation study, both algorithms show outstanding performance in tackling conservatism issue that Discounted LinUCB struggles with.
Comments: In Proceedings of the 36th Annual Conference on Uncertainty in Artificial Intelligence, 2020. (UAI 2020)
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)
Cite as: arXiv:1912.05695 [stat.ML]
  (or arXiv:1912.05695v4 [stat.ML] for this version)

Submission history

From: Baekjin Kim [view email]
[v1] Wed, 11 Dec 2019 23:34:12 GMT (49kb)
[v2] Wed, 15 Jan 2020 00:33:57 GMT (49kb)
[v3] Thu, 20 Feb 2020 18:40:14 GMT (1137kb,D)
[v4] Fri, 12 Jun 2020 17:04:55 GMT (1159kb,D)

Link back to: arXiv, form interface, contact.