Bayesian Experience Reuse for Learning from Multiple Demonstrators
  Learning from demonstrations (LfD) improves the exploration efficiency of a
learning agent by incorporating demonstrations from experts. However,
demonstration data can often come from multiple experts with conflicting goals,
making it difficult to incorporate safely and effectively in online settings.
We address this problem in the static and dynamic optimization settings by
modelling the uncertainty in source and target task functions using
normal-inverse-gamma priors, whose corresponding posteriors are, respectively,
learned from demonstrations and target data using Bayesian neural networks with
shared features. We use this learned belief to derive a quadratic programming
problem whose solution yields a probability distribution over the expert
models. Finally, we propose Bayesian Experience Reuse (BERS) to sample
demonstrations in accordance with this distribution and reuse them directly in
new tasks. We demonstrate the effectiveness of this approach for static
optimization of smooth functions, and transfer learning in a high-dimensional
supply chain problem with cost uncertainty.
