We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

math.OC

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Optimization and Control

Title: Fixed-point equations solving Risk-sensitive MDP with constraint

Abstract: There are no computationally feasible algorithms that provide solutions to the finite horizon Risk-sensitive Constrained Markov Decision Process (Risk-CMDP) problem, even for problems with moderate horizon. With an aim to design the same, we derive a fixed-point equation such that the optimal policy of Risk-CMDP is also a solution. We further provide two optimization problems equivalent to the Risk-CMDP. These formulations are instrumental in designing a global algorithm that converges to the optimal policy. The proposed algorithm is based on random restarts and a local improvement step, where the local improvement step utilizes the solution of the derived fixed-point equation; random restarts ensure global optimization. We also provide numerical examples to illustrate the feasibility of our algorithm for inventory control problem with risk-sensitive cost and constraint. The complexity of the algorithm grows only linearly with the time-horizon.
Comments: 8 pages, 4 figures, submitted to the 2023 American Control Conference (ACC)
Subjects: Optimization and Control (math.OC)
Cite as: arXiv:2210.02686 [math.OC]
  (or arXiv:2210.02686v1 [math.OC] for this version)

Submission history

From: Vartika Singh [view email]
[v1] Thu, 6 Oct 2022 05:31:56 GMT (523kb,D)

Link back to: arXiv, form interface, contact.