Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Differentiable Meta-Learning in Contextual Bandits
(Submitted on 9 Jun 2020 (this version), latest version 6 Jan 2021 (v2))
Abstract: We study a contextual bandit setting where the learning agent has access to sampled bandit instances from an unknown prior distribution $\mathcal{P}$. The goal of the agent is to achieve high reward on average over the instances drawn from $\mathcal{P}$. This setting is of a particular importance because it formalizes the offline optimization of bandit policies, to perform well on average over anticipated bandit instances. The main idea in our work is to optimize differentiable bandit policies by policy gradients. We derive reward gradients that reflect the structure of our problem, and propose contextual policies that are parameterized in a differentiable way and have low regret. Our algorithmic and theoretical contributions are supported by extensive experiments that show the importance of baseline subtraction, learned biases, and the practicality of our approach on a range of classification tasks.
Submission history
From: Branislav Kveton [view email][v1] Tue, 9 Jun 2020 07:45:41 GMT (5045kb,D)
[v2] Wed, 6 Jan 2021 03:44:39 GMT (9323kb,D)
Link back to: arXiv, form interface, contact.