Learning Transition Models with Time-delayed Causal Relations
  This paper introduces an algorithm for discovering implicit and delayed
causal relations between events observed by a robot at arbitrary times, with
the objective of improving data-efficiency and interpretability of model-based
reinforcement learning (RL) techniques. The proposed algorithm initially
predicts observations with the Markov assumption, and incrementally introduces
new hidden variables to explain and reduce the stochasticity of the
observations. The hidden variables are memory units that keep track of
pertinent past events. Such events are systematically identified by their
information gains. The learned transition and reward models are then used for
planning. Experiments on simulated and real robotic tasks show that this method
significantly improves over current RL techniques.
