Title: An axiomatic approach to Markov decision problems

Abstract: This paper presents an axiomatic approach to Markov decision problems where the discount rate is zero. The main results of the paper provide preference foundations for $0$-discount optimality and average overtaking optimality in Markov decision problems with finitely many states and finitely many actions. These results have implications for disciplines where dynamic programming problems arise, including automatic control, dynamic games, and economic development.