Title: Quantum Algorithms for Solving Dynamic Programming Problems

Abstract: We present quantum algorithms for solving finite-horizon and infinite-horizon dynamic programming problems. The infinite-horizon problems are studied using the framework of Markov decision processes. We prove query complexity lower bounds for classical randomized algorithms for the same tasks and consequently demonstrate a polynomial separation between the query complexity of our quantum algorithms and best-case query complexity of classical randomized algorithms. Up to polylogarithmic factors, our quantum algorithms provide quadratic advantage in terms of the number of states $|S|$, and the number of actions $|A|$, in the Markov decision process when the transition kernels are deterministic. This covers all discrete and combinatorial optimization problems solved classically using dynamic programming techniques. In particular, we show that our quantum algorithm solves the travelling salesperson problem in $O^*(c^4 \sqrt{2^n})$ where $n$ is the number of nodes of the underlying graph and $c$ is the maximum edge-weight of it. For stochastic transition kernels the quantum advantage is again quadratic in terms of the numbers of actions but less than quadratic (from $|S|^2$ to $|S|^{3/2}$) in terms of the numbers of states. In all cases, the speed-up achieved is at the expense of appearance of other polynomial factors in the scaling of the algorithm. Finally we prove lower bounds for the query complexity of our quantum algorithms and show that no more-than-quadratic speed-up in either of $|S|$ or $|A|$ can be achieved for solving dynamic programming and Markov decision problems using quantum algorithms.