Uncertainty Aware System Identification with Universal Policies
  Sim2real transfer is primarily concerned with transferring policies trained
in simulation to potentially noisy real world environments. A common problem
associated with sim2real transfer is estimating the real-world environmental
parameters to ground the simulated environment to. Although existing methods
such as Domain Randomisation (DR) can produce robust policies by sampling from
a distribution of parameters during training, there is no established method
for identifying the parameters of the corresponding distribution for a given
real-world setting. In this work, we propose Uncertainty-aware policy search
(UncAPS), where we use Universal Policy Network (UPN) to store
simulation-trained task-specific policies across the full range of
environmental parameters and then subsequently employ robust Bayesian
optimisation to craft robust policies for the given environment by combining
relevant UPN policies in a DR like fashion. Such policy-driven grounding is
expected to be more efficient as it estimates only task-relevant sets of
parameters. Further, we also account for the estimation uncertainties in the
search process to produce policies that are robust against both aleatoric and
epistemic uncertainties. We empirically evaluate our approach in a range of
noisy, continuous control environments, and show its improved performance
compared to competing baselines.
