Provably Efficient Representation Learning in Low-rank Markov Decision
  Processes
  The success of deep reinforcement learning (DRL) is due to the power of
learning a representation that is suitable for the underlying exploration and
exploitation task. However, existing provable reinforcement learning algorithms
with linear function approximation often assume the feature representation is
known and fixed. In order to understand how representation learning can improve
the efficiency of RL, we study representation learning for a class of low-rank
Markov Decision Processes (MDPs) where the transition kernel can be represented
in a bilinear form. We propose a provably efficient algorithm called ReLEX that
can simultaneously learn the representation and perform exploration. We show
that ReLEX always performs no worse than a state-of-the-art algorithm without
representation learning, and will be strictly better in terms of sample
efficiency if the function class of representations enjoys a certain mild
"coverage'' property over the whole state-action space.
