Planning in Markov Decision Processes with Gap-Dependent Sample
  Complexity
  We propose MDP-GapE, a new trajectory-based Monte-Carlo Tree Search algorithm
for planning in a Markov Decision Process in which transitions have a finite
support. We prove an upper bound on the number of calls to the generative
models needed for MDP-GapE to identify a near-optimal action with high
probability. This problem-dependent sample complexity result is expressed in
terms of the sub-optimality gaps of the state-action pairs that are visited
during exploration. Our experiments reveal that MDP-GapE is also effective in
practice, in contrast with other algorithms with sample complexity guarantees
in the fixed-confidence setting, that are mostly theoretical.
