We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

eess.SY

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Electrical Engineering and Systems Science > Systems and Control

Title: Comparison of Deep Reinforcement Learning and Model Predictive Control for Adaptive Cruise Control

Abstract: This study compares Deep Reinforcement Learning (DRL) and Model Predictive Control (MPC) for Adaptive Cruise Control (ACC) design in car-following scenarios. A first-order system is used as the Control-Oriented Model (COM) to approximate the acceleration command dynamics of a vehicle. Based on the equations of the control system and the multi-objective cost function, we train a DRL policy using Deep Deterministic Policy Gradient (DDPG) and solve the MPC problem via Interior-Point Optimization (IPO). Simulation results for the episode costs show that, when there are no modeling errors and the testing inputs are within the training data range, the DRL solution is equivalent to MPC with a sufficiently long prediction horizon. Particularly, the DRL episode cost is only 5.8% higher than the benchmark solution provided by optimizing the entire episode via IPO. The DRL control performance degrades when the testing inputs are outside the training data range, indicating inadequate generalization. When there are modeling errors due to control delays, disturbances, and/or testing with a High-Fidelity Model (HFM) of the vehicle, the DRL-trained policy performs better with large modeling errors while having similar performance as MPC when the modeling errors are small.
Subjects: Systems and Control (eess.SY)
DOI: 10.1109/TIV.2020.3012947
Cite as: arXiv:1910.12047 [eess.SY]
  (or arXiv:1910.12047v3 [eess.SY] for this version)

Submission history

From: Yuan Lin [view email]
[v1] Sat, 26 Oct 2019 11:15:11 GMT (1612kb)
[v2] Fri, 20 Dec 2019 19:22:02 GMT (1600kb)
[v3] Wed, 20 May 2020 13:59:31 GMT (1604kb)

Link back to: arXiv, form interface, contact.