Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Model selection for contextual bandits
(Submitted on 3 Jun 2019 (this version), latest version 14 Nov 2019 (v3))
Abstract: We introduce the problem of model selection for contextual bandits, wherein a learner must adapt to the complexity of the optimal policy while balancing exploration and exploitation. Our main result is a new model selection guarantee for linear contextual bandits. We work in the stochastic realizable setting with a sequence of nested linear policy classes of dimension $d_1 < d_2 < \ldots$, where the $m^\star$-th class contains the optimal policy, and we design an algorithm that achieves $\tilde{O}(T^{2/3}d^{1/3}_{m^\star})$ regret with no prior knowledge of the optimal dimension $d_{m^\star}$. The algorithm also achieves regret $\tilde{O}(T^{3/4} + \sqrt{Td_{m^\star}})$, which is optimal for $d_{m^{\star}}\geq{}\sqrt{T}$. This is the first contextual bandit model selection result with non-vacuous regret for all values of $d_{m^\star}$ and, to the best of our knowledge, is the first guarantee of its type in any contextual bandit setting. The core of the algorithm is a new estimator for the gap in best loss achievable by two linear policy classes, which we show admits a convergence rate faster than what is required to learn either class.
Submission history
From: Dylan Foster [view email][v1] Mon, 3 Jun 2019 02:28:45 GMT (43kb)
[v2] Sun, 27 Oct 2019 11:05:11 GMT (82kb,D)
[v3] Thu, 14 Nov 2019 06:08:57 GMT (82kb,D)
Link back to: arXiv, form interface, contact.