We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:


Current browse context:


Change to browse by:

References & Citations

DBLP - CS Bibliography


(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: Provably Efficient Multi-Agent Reinforcement Learning with Fully Decentralized Communication

Abstract: A challenge in reinforcement learning (RL) is minimizing the cost of sampling associated with exploration. Distributed exploration reduces sampling complexity in multi-agent RL (MARL). We investigate the benefits to performance in MARL when exploration is fully decentralized. Specifically, we consider a class of online, episodic, tabular $Q$-learning problems under time-varying reward and transition dynamics, in which agents can communicate in a decentralized manner.We show that group performance, as measured by the bound on regret, can be significantly improved through communication when each agent uses a decentralized message-passing protocol, even when limited to sending information up to its $\gamma$-hop neighbors. We prove regret and sample complexity bounds that depend on the number of agents, communication network structure and $\gamma.$ We show that incorporating more agents and more information sharing into the group learning scheme speeds up convergence to the optimal policy. Numerical simulations illustrate our results and validate our theoretical claims.
Comments: Accepted as a conference paper to American Control Conference (ACC) 2022
Subjects: Machine Learning (cs.LG); Multiagent Systems (cs.MA); Optimization and Control (math.OC)
Cite as: arXiv:2110.07392 [cs.LG]
  (or arXiv:2110.07392v2 [cs.LG] for this version)

Submission history

From: Justin Lidard [view email]
[v1] Thu, 14 Oct 2021 14:27:27 GMT (1343kb,D)
[v2] Mon, 2 May 2022 16:28:39 GMT (3829kb,D)

Link back to: arXiv, form interface, contact.