Current browse context:
stat.ML
Change to browse by:
References & Citations
Statistics > Machine Learning
Title: Stochastic Bandits with Delay-Dependent Payoffs
(Submitted on 7 Oct 2019 (v1), last revised 19 Feb 2020 (this version, v4))
Abstract: Motivated by recommendation problems in music streaming platforms, we propose a nonstationary stochastic bandit model in which the expected reward of an arm depends on the number of rounds that have passed since the arm was last pulled. After proving that finding an optimal policy is NP-hard even when all model parameters are known, we introduce a class of ranking policies provably approximating, to within a constant factor, the expected reward of the optimal policy. We show an algorithm whose regret with respect to the best ranking policy is bounded by $\widetilde{\mathcal{O}}\big(\!\sqrt{kT}\big)$, where $k$ is the number of arms and $T$ is time. Our algorithm uses only $\mathcal{O}\big(k\ln\ln T\big)$ switches, which helps when switching between policies is costly. As constructing the class of learning policies requires ordering the arms according to their expectations, we also bound the number of pulls required to do so. Finally, we run experiments to compare our algorithm against UCB on different problem instances.
Submission history
From: Leonardo Cella [view email][v1] Mon, 7 Oct 2019 12:48:39 GMT (263kb,D)
[v2] Wed, 9 Oct 2019 18:28:04 GMT (418kb,D)
[v3] Fri, 11 Oct 2019 19:20:10 GMT (264kb,D)
[v4] Wed, 19 Feb 2020 10:38:37 GMT (260kb,D)
Link back to: arXiv, form interface, contact.