BADDr: Bayes-Adaptive Deep Dropout RL for POMDPs
  While reinforcement learning (RL) has made great advances in scalability,
exploration and partial observability are still active research topics. In
contrast, Bayesian RL (BRL) provides a principled answer to both state
estimation and the exploration-exploitation trade-off, but struggles to scale.
To tackle this challenge, BRL frameworks with various prior assumptions have
been proposed, with varied success. This work presents a
representation-agnostic formulation of BRL under partially observability,
unifying the previous models under one theoretical umbrella. To demonstrate its
practical significance we also propose a novel derivation, Bayes-Adaptive Deep
Dropout rl (BADDr), based on dropout networks. Under this parameterization, in
contrast to previous work, the belief over the state and dynamics is a more
scalable inference problem. We choose actions through Monte-Carlo tree search
and empirically show that our method is competitive with state-of-the-art BRL
methods on small domains while being able to solve much larger ones.
