Current browse context:
cs.AI
Change to browse by:
References & Citations
Computer Science > Artificial Intelligence
Title: Reinforcement Learning of POMDPs using Spectral Methods
(Submitted on 25 Feb 2016 (v1), last revised 29 May 2016 (this version, v2))
Abstract: We propose a new reinforcement learning algorithm for partially observable Markov decision processes (POMDP) based on spectral decomposition methods. While spectral methods have been previously employed for consistent learning of (passive) latent variable models such as hidden Markov models, POMDPs are more challenging since the learner interacts with the environment and possibly changes the future observations in the process. We devise a learning algorithm running through episodes, in each episode we employ spectral techniques to learn the POMDP parameters from a trajectory generated by a fixed policy. At the end of the episode, an optimization oracle returns the optimal memoryless planning policy which maximizes the expected reward based on the estimated POMDP model. We prove an order-optimal regret bound with respect to the optimal memoryless policy and efficient scaling with respect to the dimensionality of observation and action spaces.
Submission history
From: Kamyar Azizzadenesheli Ph.D. [view email][v1] Thu, 25 Feb 2016 01:25:36 GMT (605kb)
[v2] Sun, 29 May 2016 07:15:21 GMT (591kb)
Link back to: arXiv, form interface, contact.