We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.IT

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Optimization and Control

Title: Transfer-Entropy-Regularized Markov Decision Processes

Abstract: We consider the framework of transfer-entropy-regularized Markov Decision Process (TERMDP) in which the weighted sum of the classical state-dependent cost and the transfer entropy from the state random process to the control random process is minimized. Although TERMDPs are generally formulated as nonconvex optimization problems, we derive an analytical necessary optimality condition expressed as a finite set of nonlinear equations, based on which an iterative forward-backward computational procedure similar to the Arimoto-Blahut algorithm is proposed. It is shown that every limit point of the sequence generated by the proposed algorithm is a stationary point of the TERMDP. Applications of TERMDPs are discussed in the context of networked control systems theory and non-equilibrium thermodynamics. The proposed algorithm is applied to an information-constrained maze navigation problem, whereby we study how the price of information qualitatively alters the optimal decision polices.
Subjects: Optimization and Control (math.OC); Information Theory (cs.IT)
Cite as: arXiv:1708.09096 [math.OC]
  (or arXiv:1708.09096v3 [math.OC] for this version)

Submission history

From: Takashi Tanaka [view email]
[v1] Wed, 30 Aug 2017 03:14:33 GMT (269kb,D)
[v2] Sat, 30 Jun 2018 20:48:21 GMT (248kb,D)
[v3] Wed, 27 May 2020 19:36:47 GMT (329kb,D)

Link back to: arXiv, form interface, contact.