Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Macro-Action-Based Deep Multi-Agent Reinforcement Learning
(Submitted on 18 Apr 2020 (v1), last revised 16 Oct 2021 (this version, v2))
Abstract: In real-world multi-robot systems, performing high-quality, collaborative behaviors requires robots to asynchronously reason about high-level action selection at varying time durations. Macro-Action Decentralized Partially Observable Markov Decision Processes (MacDec-POMDPs) provide a general framework for asynchronous decision making under uncertainty in fully cooperative multi-agent tasks. However, multi-agent deep reinforcement learning methods have only been developed for (synchronous) primitive-action problems. This paper proposes two Deep Q-Network (DQN) based methods for learning decentralized and centralized macro-action-value functions with novel macro-action trajectory replay buffers introduced for each case. Evaluations on benchmark problems and a larger domain demonstrate the advantage of learning with macro-actions over primitive-actions and the scalability of our approaches.
Submission history
From: Yuchen Xiao [view email][v1] Sat, 18 Apr 2020 15:46:38 GMT (4834kb,D)
[v2] Sat, 16 Oct 2021 19:01:41 GMT (4849kb,D)
Link back to: arXiv, form interface, contact.