References & Citations
Mathematics > Optimization and Control
Title: Approximating the Stationary Bellman Equation by Hierarchical Tensor Products
(Submitted on 1 Nov 2019 (v1), last revised 18 May 2021 (this version, v4))
Abstract: We treat infinite horizon optimal control problems by solving the associated stationary Hamilton-Jacobi-Bellman (HJB) equation numerically to compute the value function and an optimal feedback law. The dynamical systems under consideration are spatial discretizations of non linear parabolic partial differential equations (PDE), which means that the HJB is non linear and suffers from the curse of dimensionality. Its non linearity is handled by the Policy Iteration algorithm, where the problem is reduced to a sequence of linear, hyperbolic PDEs. These equations remain the computational bottleneck due to their high dimensions. By the method of characteristics these linearized HJB equations can be reformulated via the Koopman operator in the spirit of dynamic programming. The resulting operator equations are solved using a minimal residual method. To overcome numerical infeasability we use low rank hierarchical tensor product approximation/tree-based tensor formats, in particular tensor trains (TT tensors), and multi-polynomials, together with high dimensional quadrature, e.g. Monte-Carlo. By controlling a destabilized version of viscous Burgers and a diffusion equation with unstable reaction term numerical evidences are given.
Submission history
From: Leon Sallandt [view email][v1] Fri, 1 Nov 2019 09:54:19 GMT (991kb)
[v2] Tue, 12 Nov 2019 11:36:37 GMT (1114kb)
[v3] Wed, 15 Apr 2020 09:59:34 GMT (758kb)
[v4] Tue, 18 May 2021 13:19:15 GMT (767kb)
Link back to: arXiv, form interface, contact.