Reinforcement Learning of POMDPs using Spectral Methods
  We propose a new reinforcement learning algorithm for partially observable
Markov decision processes (POMDP) based on spectral decomposition methods.
While spectral methods have been previously employed for consistent learning of
(passive) latent variable models such as hidden Markov models, POMDPs are more
challenging since the learner interacts with the environment and possibly
changes the future observations in the process. We devise a learning algorithm
running through episodes, in each episode we employ spectral techniques to
learn the POMDP parameters from a trajectory generated by a fixed policy. At
the end of the episode, an optimization oracle returns the optimal memoryless
planning policy which maximizes the expected reward based on the estimated
POMDP model. We prove an order-optimal regret bound with respect to the optimal
memoryless policy and efficient scaling with respect to the dimensionality of
observation and action spaces.
