Provably Efficient Reward-Agnostic Navigation with Linear Value
  Iteration
  There has been growing progress on theoretical analyses for provably
efficient learning in MDPs with linear function approximation, but much of the
existing work has made strong assumptions to enable exploration by conventional
exploration frameworks. Typically these assumptions are stronger than what is
needed to find good solutions in the batch setting. In this work, we show how
under a more standard notion of low inherent Bellman error, typically employed
in least-square value iteration-style algorithms, we can provide strong PAC
guarantees on learning a near optimal value function provided that the linear
space is sufficiently "explorable". We present a computationally tractable
algorithm for the reward-free setting and show how it can be used to learn a
near optimal policy for any (linear) reward function, which is revealed only
once learning has completed. If this reward function is also estimated from the
samples gathered during pure exploration, our results also provide same-order
PAC guarantees on the performance of the resulting policy for this setting.
