Current browse context:
stat.ML
Change to browse by:
References & Citations
Statistics > Machine Learning
Title: Online Action Learning in High Dimensions: A Conservative Perspective
(Submitted on 29 Sep 2020 (v1), last revised 23 Mar 2024 (this version, v4))
Abstract: Sequential learning problems are common in several fields of research and practical applications. Examples include dynamic pricing and assortment, design of auctions and incentives and permeate a large number of sequential treatment experiments. In this paper, we extend one of the most popular learning solutions, the $\epsilon_t$-greedy heuristics, to high-dimensional contexts considering a conservative directive. We do this by allocating part of the time the original rule uses to adopt completely new actions to a more focused search in a restrictive set of promising actions. The resulting rule might be useful for practical applications that still values surprises, although at a decreasing rate, while also has restrictions on the adoption of unusual actions. With high probability, we find reasonable bounds for the cumulative regret of a conservative high-dimensional decaying $\epsilon_t$-greedy rule. Also, we provide a lower bound for the cardinality of the set of viable actions that implies in an improved regret bound for the conservative version when compared to its non-conservative counterpart. Additionally, we show that end-users have sufficient flexibility when establishing how much safety they want, since it can be tuned without impacting theoretical properties. We illustrate our proposal both in a simulation exercise and using a real dataset.
Submission history
From: Marcelo Medeiros [view email][v1] Tue, 29 Sep 2020 12:25:05 GMT (279kb)
[v2] Mon, 9 Nov 2020 03:15:00 GMT (137kb,D)
[v3] Mon, 25 Oct 2021 21:46:53 GMT (506kb,D)
[v4] Sat, 23 Mar 2024 15:01:03 GMT (0kb,I)
Link back to: arXiv, form interface, contact.