References & Citations
Computer Science > Machine Learning
Title: Gambler's Ruin Bandit Problem
(Submitted on 21 May 2016 (this version), latest version 29 Sep 2016 (v3))
Abstract: In this paper, we propose a new sequential decision making problem called {\em gambler's ruin bandit problem} (GRBP). In each round of the GRBP the learner faces a gambler's ruin problem with two possible actions: a {\em continuation action} that moves the learner randomly over the state space around the current state; and a {\em terminal action} that moves the learner directly into one of the two terminal states (goal and dead-end state). The current round ends when a terminal state is reached. We first formulate GRBP as an optimization problem, and prove that the optimal policy is characterized by a simple threshold rule. The problem is solved for infinite time budget. Then, we consider the case when the state transition probabilities are unknown and provide logarithmic problem specific regret bounds. We also identify a condition under which the learner only incurs finite regret. Numerous applications including optimal medical treatment assignment can be formulated as a GRBP, in which the continuation action corresponds to the conservative treatment and the terminal action corresponds to the surgery.
Submission history
From: Nima Akbarzadeh [view email][v1] Sat, 21 May 2016 14:43:44 GMT (273kb,D)
[v2] Mon, 11 Jul 2016 13:54:20 GMT (416kb,D)
[v3] Thu, 29 Sep 2016 03:51:57 GMT (504kb,D)
Link back to: arXiv, form interface, contact.