Imitation Learning from Pixel-Level Demonstrations by HashReward
  One of the key issues for imitation learning lies in making policy learned
from limited samples to generalize well in the whole state-action space. This
problem is much more severe in high-dimensional state environments, such as
game playing with raw pixel inputs. Under this situation, even state-of-the-art
adversary-based imitation learning algorithms fail. Through empirical studies,
we find that the main cause lies in the failure of training a powerful
discriminator to generate meaningful rewards in high-dimensional environments.
Although it seems that dimensionality reduction can help, a straightforward
application of off-the-shelf methods cannot achieve good performance. In this
work, we show in theory that the balance between dimensionality reduction and
discriminative training is essential for effective learning. To achieve this
target, we propose HashReward, which utilizes the idea of supervised hashing to
realize such an ideal balance. Experimental results show that HashReward could
outperform state-of-the-art methods for a large gap under the challenging
high-dimensional environments.
