We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.LG

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: MAVIPER: Learning Decision Tree Policies for Interpretable Multi-Agent Reinforcement Learning

Abstract: Many recent breakthroughs in multi-agent reinforcement learning (MARL) require the use of deep neural networks, which are challenging for human experts to interpret and understand. On the other hand, existing work on interpretable RL has shown promise in extracting more interpretable decision tree-based policies, but only in the single-agent setting. To fill this gap, we propose the first set of interpretable MARL algorithms that extract decision-tree policies from neural networks trained with MARL. The first algorithm, IVIPER, extends VIPER, a recent method for single-agent interpretable RL, to the multi-agent setting. We demonstrate that IVIPER can learn high-quality decision-tree policies for each agent. To better capture coordination between agents, we propose a novel centralized decision-tree training algorithm, MAVIPER. MAVIPER jointly grows the trees of each agent by predicting the behavior of the other agents using their anticipated trees, and uses resampling to focus on states that are critical for its interactions with other agents. We show that both algorithms generally outperform the baselines and that MAVIPER-trained agents achieve better-coordinated performance than IVIPER-trained agents on three different multi-agent particle-world environments.
Comments: 25 pages
Subjects: Machine Learning (cs.LG); Multiagent Systems (cs.MA)
Cite as: arXiv:2205.12449 [cs.LG]
  (or arXiv:2205.12449v1 [cs.LG] for this version)

Submission history

From: Stephanie Milani [view email]
[v1] Wed, 25 May 2022 02:38:10 GMT (1234kb,D)
[v2] Mon, 11 Jul 2022 20:46:36 GMT (9233kb,D)

Link back to: arXiv, form interface, contact.