Selecting the State-Representation in Reinforcement Learning
  The problem of selecting the right state-representation in a reinforcement
learning problem is considered. Several models (functions mapping past
observations to a finite set) of the observations are given, and it is known
that for at least one of these models the resulting state dynamics are indeed
Markovian. Without knowing neither which of the models is the correct one, nor
what are the probabilistic characteristics of the resulting MDP, it is required
to obtain as much reward as the optimal policy for the correct model (or for
the best of the correct models, if there are several). We propose an algorithm
that achieves that, with a regret of order T^{2/3} where T is the horizon time.
