Current browse context:
math.OC
Change to browse by:
References & Citations
Mathematics > Optimization and Control
Title: Scalable Reinforcement Learning for Multi-Agent Networked Systems
(Submitted on 5 Dec 2019 (v1), last revised 1 Nov 2021 (this version, v3))
Abstract: We study reinforcement learning (RL) in a setting with a network of agents whose states and actions interact in a local manner where the objective is to find localized policies such that the (discounted) global reward is maximized. A fundamental challenge in this setting is that the state-action space size scales exponentially in the number of agents, rendering the problem intractable for large networks. In this paper, we propose a Scalable Actor Critic (SAC) framework that exploits the network structure and finds a localized policy that is an $O(\rho^{\kappa})$-approximation of a stationary point of the objective for some $\rho\in(0,1)$, with complexity that scales with the local state-action space size of the largest $\kappa$-hop neighborhood of the network. We illustrate our model and approach using examples from wireless communication, epidemics and traffic.
Submission history
From: Guannan Qu [view email][v1] Thu, 5 Dec 2019 22:44:07 GMT (61kb)
[v2] Tue, 18 Feb 2020 19:42:18 GMT (629kb,D)
[v3] Mon, 1 Nov 2021 02:10:15 GMT (249kb,D)
Link back to: arXiv, form interface, contact.