We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

eess.SY

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Electrical Engineering and Systems Science > Systems and Control

Title: Ensemble Kalman Filter (EnKF) for Reinforcement Learning (RL)

Abstract: This paper is concerned with the problem of representing and learning the optimal control law for the linear quadratic Gaussian (LQG) optimal control problem. In recent years, there is a growing interest in re-visiting this classical problem, in part due to the successes of reinforcement learning (RL). The main question of this body of research (and also of our paper) is to approximate the optimal control law {\em without} explicitly solving the Riccati equation. For this purpose, a novel simulation-based algorithm, namely an ensemble Kalman filter (EnKF), is introduced in this paper. The algorithm is used to obtain formulae for optimal control, expressed entirely in terms of the EnKF particles. For the general partially observed LQG problem, the proposed EnKF is combined with a standard EnKF (for the estimation problem) to obtain the optimal control input based on the use of the separation principle. A nonlinear extension of the algorithm is also discussed which clarifies the duality roots of the proposed EnKF. The theoretical results and algorithms are illustrated with numerical experiments.
Subjects: Systems and Control (eess.SY); Optimization and Control (math.OC)
Cite as: arXiv:2107.01244 [eess.SY]
  (or arXiv:2107.01244v1 [eess.SY] for this version)

Submission history

From: Amirhossein Taghvaei [view email]
[v1] Fri, 2 Jul 2021 19:29:52 GMT (199kb,D)

Link back to: arXiv, form interface, contact.