Efficient Empowerment Estimation for Unsupervised Stabilization
  Intrinsically motivated artificial agents learn advantageous behavior without
externally-provided rewards. Previously, it was shown that maximizing mutual
information between agent actuators and future states, known as the empowerment
principle, enables unsupervised stabilization of dynamical systems at upright
positions, which is a prototypical intrinsically motivated behavior for upright
standing and walking. This follows from the coincidence between the objective
of stabilization and the objective of empowerment. Unfortunately, sample-based
estimation of this kind of mutual information is challenging. Recently, various
variational lower bounds (VLBs) on empowerment have been proposed as solutions;
however, they are often biased, unstable in training, and have high sample
complexity. In this work, we propose an alternative solution based on a
trainable representation of a dynamical system as a Gaussian channel, which
allows us to efficiently calculate an unbiased estimator of empowerment by
convex optimization. We demonstrate our solution for sample-based unsupervised
stabilization on different dynamical control systems and show the advantages of
our method by comparing it to the existing VLB approaches. Specifically, we
show that our method has a lower sample complexity, is more stable in training,
possesses the essential properties of the empowerment function, and allows
estimation of empowerment from images. Consequently, our method opens a path to
wider and easier adoption of empowerment for various applications.
