Error-Aware Policy Learning: Zero-Shot Generalization in Partially
  Observable Dynamic Environments
  Simulation provides a safe and efficient way to generate useful data for
learning complex robotic tasks. However, matching simulation and real-world
dynamics can be quite challenging, especially for systems that have a large
number of unobserved or unmeasurable parameters, which may lie in the robot
dynamics itself or in the environment with which the robot interacts. We
introduce a novel approach to tackle such a sim-to-real problem by developing
policies capable of adapting to new environments, in a zero-shot manner. Key to
our approach is an error-aware policy (EAP) that is explicitly made aware of
the effect of unobservable factors during training. An EAP takes as input the
predicted future state error in the target environment, which is provided by an
error-prediction function, simultaneously trained with the EAP. We validate our
approach on an assistive walking device trained to help the human user recover
from external pushes. We show that a trained EAP for a hip-torque assistive
device can be transferred to different human agents with unseen biomechanical
characteristics. In addition, we show that our method can be applied to other
standard RL control tasks.
