Bayesian Policy Search for Stochastic Domains
  AI planning can be cast as inference in probabilistic models, and
probabilistic programming was shown to be capable of policy search in partially
observable domains. Prior work introduces policy search through Markov chain
Monte Carlo in deterministic domains, as well as adapts black-box variational
inference to stochastic domains, however not in the strictly Bayesian sense. In
this work, we cast policy search in stochastic domains as a Bayesian inference
problem and provide a scheme for encoding such problems as nested probabilistic
programs. We argue that probabilistic programs for policy search in stochastic
domains should involve nested conditioning, and provide an adaption of
Lightweight Metropolis-Hastings (LMH) for robust inference in such programs. We
apply the proposed scheme to stochastic domains and show that policies of
similar quality are learned, despite a simpler and more general inference
algorithm. We believe that the proposed variant of LMH is novel and applicable
to a wider class of probabilistic programs with nested conditioning.
