References & Citations
Computer Science > Artificial Intelligence
Title: ReZero: Boosting MCTS-based Algorithms by Just-in-Time and Speedy Reanalyze
(Submitted on 25 Apr 2024 (v1), last revised 28 Apr 2024 (this version, v2))
Abstract: MCTS-based algorithms, such as MuZero and its derivatives, have achieved widespread success in various decision-making domains. These algorithms employ the reanalyze process to enhance sample efficiency, albeit at the expense of significant wall-clock time consumption. To address this issue, we propose a general approach named ReZero to boost MCTS-based algorithms. Specifically, we propose a new scheme that simplifies data collecting and reanalyzing, which significantly reduces the search cost while guarantees the performance as well. Furthermore, to accelerate each search process, we conceive a method to reuse the subsequent information in the trajectory. The corresponding analysis conducted on the bandit model also provides auxiliary theoretical substantiation for our design. Experiments conducted on Atari environments and board games demonstrates that ReZero substantially improves training speed while maintaining high sample efficiency. The code is available as part of the LightZero benchmark at this https URL
Submission history
From: Chunyu Xuan [view email][v1] Thu, 25 Apr 2024 07:02:07 GMT (1569kb,D)
[v2] Sun, 28 Apr 2024 06:21:04 GMT (1570kb,D)
Link back to: arXiv, form interface, contact.