We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:


Current browse context:


Change to browse by:

References & Citations


(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Optimization and Control

Title: Transfer-Entropy-Regularized Markov Decision Processes

Abstract: We consider the framework of transfer-entropy-regularized Markov Decision Process (TERMDP) in which the weighted sum of the classical state-dependent cost and the transfer entropy from the state random process to the control random process is minimized. Although TERMDP is generally a nonconvex optimization problem, we derive an analytical necessary optimality condition expressed as a finite set of nonlinear equations, based on which an iterative forward-backward computational procedure similar to the Arimoto-Blahut algorithm is proposed. Convergence of the proposed algorithm to a stationary point of the considered TERMDP is established. Applications of TERMDP are discussed in the context of networked control systems theory and non-equilibrium thermodynamics. The proposed algorithm is applied to an information-constrained maze navigation problem, whereby we study how the price of information qualitatively alters the optimal decision polices.
Subjects: Optimization and Control (math.OC); Information Theory (cs.IT)
Cite as: arXiv:1708.09096 [math.OC]
  (or arXiv:1708.09096v2 [math.OC] for this version)

Submission history

From: Takashi Tanaka [view email]
[v1] Wed, 30 Aug 2017 03:14:33 GMT (269kb,D)
[v2] Sat, 30 Jun 2018 20:48:21 GMT (248kb,D)
[v3] Wed, 27 May 2020 19:36:47 GMT (329kb,D)

Link back to: arXiv, form interface, contact.