We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

math.OC

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Optimization and Control

Title: Finite State Markov Decision Processes with Transfer Entropy Costs

Abstract: We consider a mathematical framework of finite state Markov Decision Processes (MDPs) in which a weighted sum of the classical state-dependent cost and the transfer entropy from the state random process to the control random process is minimized. Physical interpretations of the considered MDPs are provided in the context of networked control systems theory and non-equilibrium thermodynamics. Based on the dynamic programming principle, we derive an optimality condition comprised of a Kolmogorov forward equation and a Bellman backward equation. As the main contribution, we propose an iterative forward-backward computational procedure similar to the Arimoto-Blahut algorithm to synthesize the optimal policy numerically. Convergence of the algorithm is established. The proposed algorithm is applied to an information-constrained navigation problem over a maze, whereby we study how the price of information alters the optimal decision polices qualitatively.
Subjects: Optimization and Control (math.OC); Information Theory (cs.IT)
Cite as: arXiv:1708.09096 [math.OC]
  (or arXiv:1708.09096v1 [math.OC] for this version)

Submission history

From: Takashi Tanaka [view email]
[v1] Wed, 30 Aug 2017 03:14:33 GMT (269kb,D)
[v2] Sat, 30 Jun 2018 20:48:21 GMT (248kb,D)
[v3] Wed, 27 May 2020 19:36:47 GMT (329kb,D)

Link back to: arXiv, form interface, contact.