We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

math.OC

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Optimization and Control

Title: Inverse Optimal Control with Discount Factor for Continuous and Discrete-Time Control-Affine Systems and Reinforcement Learning

Abstract: This paper addresses the inverse optimal control problem of finding the state weighting function that leads to a quadratic value function when the cost on the input is fixed to be quadratic. The paper focuses on a class of infinite horizon discrete-time and continuous-time optimal control problems whose dynamics are control-affine and whose cost is quadratic in the input. The optimal control policy for this problem is the projection of minus the gradient of the value function onto the space formed by all feasible control directions. This projection points along the control direction of steepest decrease of the value function. For discrete-time systems and a quadratic value function the optimal control law can be obtained as the solution of a regularized least squares program, which corresponds to a receding horizon control with a single step ahead. For the single input case and a quadratic value function the solution for small weights in the control energy is interpreted as a control policy that at each step brings the trajectories of the system as close as possible to the origin, as measured by an appropriate norm. Conditions under which the optimal control law is linear are also stated. Additionally, the paper offers a mapping of the optimal control formulation to an equivalent reinforcement learning formulation. Examples show the application of the theoretical results.
Comments: published at IEEE Conference on Decision and Control 2022
Subjects: Optimization and Control (math.OC); Systems and Control (eess.SY)
Cite as: arXiv:2211.09917 [math.OC]
  (or arXiv:2211.09917v1 [math.OC] for this version)

Submission history

From: Luis Rodrigues [view email]
[v1] Thu, 17 Nov 2022 22:30:12 GMT (157kb,D)

Link back to: arXiv, form interface, contact.