Safe Interactive Model-Based Learning
  Control applications present hard operational constraints. A violation of
these can result in unsafe behavior. This paper introduces Safe Interactive
Model Based Learning (SiMBL), a framework to refine an existing controller and
a system model while operating on the real environment. SiMBL is composed of
the following trainable components: a Lyapunov function, which determines a
safe set; a safe control policy; and a Bayesian RNN forward model. A min-max
control framework, based on alternate minimisation and backpropagation through
the forward model, is used for the offline computation of the controller and
the safe set. Safety is formally verified a-posteriori with a probabilistic
method that utilizes the Noise Contrastive Priors (NPC) idea to build a
Bayesian RNN forward model with an additive state uncertainty estimate which is
large outside the training data distribution. Iterative refinement of the model
and the safe set is achieved thanks to a novel loss that conditions the
uncertainty estimates of the new model to be close to the current one. The
learned safe set and model can also be used for safe exploration, i.e., to
collect data within the safe invariant set, for which a simple one-step MPC is
proposed. The single components are tested on the simulation of an inverted
pendulum with limited torque and stability region, showing that iteratively
adding more data can improve the model, the controller and the size of the safe
region.
