We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

eess.SY

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Electrical Engineering and Systems Science > Systems and Control

Title: Robust Model Predictive Shielding for Safe Reinforcement Learning with Stochastic Dynamics

Abstract: This paper proposes a framework for safe reinforcement learning that can handle stochastic nonlinear dynamical systems. We focus on the setting where the nominal dynamics are known, and are subject to additive stochastic disturbances with known distribution. Our goal is to ensure the safety of a control policy trained using reinforcement learning, e.g., in a simulated environment. We build on the idea of model predictive shielding (MPS), where a backup controller is used to override the learned policy as needed to ensure safety. The key challenge is how to compute a backup policy in the context of stochastic dynamics. We propose to use a tube-based robust NMPC controller as the backup controller. We estimate the tubes using sampled trajectories, leveraging ideas from statistical learning theory to obtain high-probability guarantees. We empirically demonstrate that our approach can ensure safety in stochastic systems, including cart-pole and a non-holonomic particle with random obstacles.
Comments: 8 pages, 5 figures, accepted by ICRA 2020
Subjects: Systems and Control (eess.SY); Machine Learning (cs.LG); Robotics (cs.RO)
Cite as: arXiv:1910.10885 [eess.SY]
  (or arXiv:1910.10885v2 [eess.SY] for this version)

Submission history

From: Shuo Li [view email]
[v1] Thu, 24 Oct 2019 02:22:42 GMT (1499kb,D)
[v2] Fri, 24 Jan 2020 03:12:03 GMT (1840kb,D)

Link back to: arXiv, form interface, contact.