Concurrent Training Improves the Performance of Behavioral Cloning from
  Observation
  Learning from demonstration is widely used as an efficient way for robots to
acquire new skills. However, it typically requires that demonstrations provide
full access to the state and action sequences. In contrast, learning from
observation offers a way to utilize unlabeled demonstrations (e.g., video) to
perform imitation learning. One approach to this is behavioral cloning from
observation (BCO). The original implementation of BCO proceeds by first
learning an inverse dynamics model and then using that model to estimate action
labels, thereby reducing the problem to behavioral cloning. However, existing
approaches to BCO require a large number of initial interactions in the first
step. Here, we provide a novel theoretical analysis of BCO, introduce a
modification BCO*, and show that in the semi-supervised setting, BCO* can
concurrently improve both its estimate for the inverse dynamics model and the
expert policy. This result allows us to eliminate the dependence on initial
interactions and dramatically improve the sample complexity of BCO. We evaluate
the effectiveness of our algorithm through experiments on various benchmark
domains. The results demonstrate that concurrent training not only improves
over the performance of BCO but also results in performance that is competitive
with state-of-the-art imitation learning methods such as GAIL and Value-Dice.
