Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Distributed TD(0) with Almost No Communication
(Submitted on 25 May 2023)
Abstract: We provide a new non-asymptotic analysis of distributed temporal difference learning with linear function approximation. Our approach relies on ``one-shot averaging,'' where $N$ agents run identical local copies of the TD(0) method and average the outcomes only once at the very end. We demonstrate a version of the linear time speedup phenomenon, where the convergence time of the distributed process is a factor of $N$ faster than the convergence time of TD(0). This is the first result proving benefits from parallelism for temporal difference methods.
Submission history
From: Alexander Olshevsky [view email][v1] Thu, 25 May 2023 17:00:46 GMT (1196kb,D)
Link back to: arXiv, form interface, contact.