We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.LG

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: Robustifying Reinforcement Learning Policies with $\mathcal{L}_1$ Adaptive Control

Abstract: A reinforcement learning (RL) policy trained in a nominal environment could fail in a new/perturbed environment due to the existence of dynamic variations. Existing robust methods try to obtain a fixed policy for all envisioned dynamic variation scenarios through robust or adversarial training. These methods could lead to conservative performance due to emphasis on the worst case, and often involve tedious modifications to the training environment. We propose an approach to robustifying a pre-trained non-robust RL policy with $\mathcal{L}_1$ adaptive control. Leveraging the capability of an $\mathcal{L}_1$ control law in the fast estimation of and active compensation for dynamic variations, our approach can significantly improve the robustness of an RL policy trained in a standard (i.e., non-robust) way, either in a simulator or in the real world. Numerical experiments are provided to validate the efficacy of the proposed approach.
Comments: A significantly extended version of this paper has been uploaded to arXiv. arXiv:2112.01953
Subjects: Machine Learning (cs.LG); Systems and Control (eess.SY)
Cite as: arXiv:2106.02249 [cs.LG]
  (or arXiv:2106.02249v4 [cs.LG] for this version)

Submission history

From: Pan Zhao [view email]
[v1] Fri, 4 Jun 2021 04:28:46 GMT (3985kb,D)
[v2] Wed, 18 Aug 2021 02:21:54 GMT (3985kb,D)
[v3] Mon, 6 Dec 2021 03:37:54 GMT (0kb,I)
[v4] Thu, 9 Dec 2021 04:20:24 GMT (3984kb,D)

Link back to: arXiv, form interface, contact.