We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ML

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Machine Learning

Title: Stochastic Bandits with Context Distributions

Abstract: We introduce a stochastic contextual bandit model where at each time step the environment chooses a distribution over a context set and samples the context from this distribution. The learner observes only the context distribution while the exact context realization remains hidden. This allows for a broad range of applications where the context is stochastic or when the learner needs to predict the context. We leverage the UCB algorithm to this setting and show that it achieves an order-optimal high-probability bound on the cumulative regret for linear and kernelized reward functions. Our results strictly generalize previous work in the sense that both our model and the algorithm reduce to the standard setting when the environment chooses only Dirac delta distributions and therefore provides the exact context to the learner. We further analyze a variant where the learner observes the realized context after choosing the action. Finally, we demonstrate the proposed method on synthetic and real-world datasets.
Comments: Accepted at NeurIPS 2019
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)
Cite as: arXiv:1906.02685 [stat.ML]
  (or arXiv:1906.02685v2 [stat.ML] for this version)

Submission history

From: Johannes Kirschner [view email]
[v1] Thu, 6 Jun 2019 16:31:18 GMT (371kb,D)
[v2] Thu, 14 Nov 2019 10:58:05 GMT (468kb,D)

Link back to: arXiv, form interface, contact.