We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cond-mat.stat-mech

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Condensed Matter > Statistical Mechanics

Title: A Tensor Network Approach to Finite Markov Decision Processes

Abstract: Tensor network (TN) techniques - often used in the context of quantum many-body physics - have shown promise as a tool for tackling machine learning (ML) problems. The application of TNs to ML, however, has mostly focused on supervised and unsupervised learning. Yet, with their direct connection to hidden Markov chains, TNs are also naturally suited to Markov decision processes (MDPs) which provide the foundation for reinforcement learning (RL). Here we introduce a general TN formulation of finite, episodic and discrete MDPs. We show how this formulation allows us to exploit algorithms developed for TNs for policy optimisation, the key aim of RL. As an application we consider the issue - formulated as an RL problem - of finding a stochastic evolution that satisfies specific dynamical conditions, using the simple example of random walk excursions as an illustration.
Comments: 10 pages, 2 figures
Subjects: Statistical Mechanics (cond-mat.stat-mech); Machine Learning (cs.LG); Quantum Physics (quant-ph)
Cite as: arXiv:2002.05185 [cond-mat.stat-mech]
  (or arXiv:2002.05185v1 [cond-mat.stat-mech] for this version)

Submission history

From: Edward Gillman [view email]
[v1] Wed, 12 Feb 2020 19:18:27 GMT (323kb,D)

Link back to: arXiv, form interface, contact.