Title: Myopic Quantal Response Policy: Thompson Sampling Meets Behavioral Economics

Abstract: We study a new family of multi-armed bandit (MAB) algorithms called Myopic Quantal Response (MQR). It prescribes a simple way to randomize over arms according to historical data and a "coefficient of exploitation," which explicitly controls the exploration-exploitation trade-off. We show that MQR partially extends the Thompson Sampling (TS) algorithm. It is also a dynamic version of quantal response models where the expected utilities are directly estimated from historical rewards. Based on theoretical analysis and numerical experiments, we believe the significance of MQR is three-fold. First, it provides a conceptual framework to understand further the exploration-exploitation trade-off. Second, it can be used as a structural estimation tool to learn from realized actions and rewards how much a given policy (either generated by human beings or algorithms) is "under" or "over" exploring. Finally, it can inspire new MAB algorithms that improve well-studied algorithms (e.g., TS) in a nonasymptotic setting.