Provable Representation Learning for Imitation with Contrastive Fourier
  Features
  In imitation learning, it is common to learn a behavior policy to match an
unknown target policy via max-likelihood training on a collected set of target
demonstrations. In this work, we consider using offline experience datasets -
potentially far from the target distribution - to learn low-dimensional state
representations that provably accelerate the sample-efficiency of downstream
imitation learning. A central challenge in this setting is that the unknown
target policy itself may not exhibit low-dimensional behavior, and so there is
a potential for the representation learning objective to alias states in which
the target policy acts differently. Circumventing this challenge, we derive a
representation learning objective that provides an upper bound on the
performance difference between the target policy and a lowdimensional policy
trained with max-likelihood, and this bound is tight regardless of whether the
target policy itself exhibits low-dimensional structure. Moving to the
practicality of our method, we show that our objective can be implemented as
contrastive learning, in which the transition dynamics are approximated by
either an implicit energy-based model or, in some special cases, an implicit
linear model with representations given by random Fourier features. Experiments
on both tabular environments and high-dimensional Atari games provide
quantitative evidence for the practical benefits of our proposed objective.
