Current browse context:
math.OC
Change to browse by:
References & Citations
Mathematics > Optimization and Control
Title: Fast Approximate Dynamic Programming for Infinite-Horizon Continuous-State Markov Decision Processes
(Submitted on 17 Feb 2021 (this version), latest version 17 Mar 2022 (v4))
Abstract: In this article, we consider the infinite-horizon, discounted cost, optimal control of discrete-time systems with separable cost and constraint in the state and input variables. Starting from deterministic linear dynamics, we introduce a novel numerical algorithm for implementation of the value iteration (VI) algorithm in the conjugate domain, using the Linear-time Legendre Transform algorithm. Detailed analyses of the convergence, complexity, and error of the proposed algorithm are provided. In particular, with a discretization of size $X$ and $U$ for the state and input spaces, respectively, the proposed approach can reduce the time complexity of each iteration of the VI algorithm from $O(XU)$ to $O(X)$, by replacing the minimization operation in the primal domain with a simple addition in the conjugate domain. Also discussed are the direct extensions of the proposed algorithm for nonlinear dynamics and stochastic dynamics with additive noise.
Submission history
From: Mohamad Amin Sharifi Kolarijani [view email][v1] Wed, 17 Feb 2021 17:19:10 GMT (86kb,D)
[v2] Wed, 26 May 2021 13:10:53 GMT (213kb,D)
[v3] Wed, 20 Oct 2021 17:58:24 GMT (251kb,D)
[v4] Thu, 17 Mar 2022 12:09:35 GMT (575kb,D)
Link back to: arXiv, form interface, contact.