Learning Retrospective Knowledge with Reverse Reinforcement Learning
  We present a Reverse Reinforcement Learning (Reverse RL) approach for
representing retrospective knowledge. General Value Functions (GVFs) have
enjoyed great success in representing predictive knowledge, i.e., answering
questions about possible future outcomes such as "how much fuel will be
consumed in expectation if we drive from A to B?". GVFs, however, cannot answer
questions like "how much fuel do we expect a car to have given it is at B at
time $t$?". To answer this question, we need to know when that car had a full
tank and how that car came to B. Since such questions emphasize the influence
of possible past events on the present, we refer to their answers as
retrospective knowledge. In this paper, we show how to represent retrospective
knowledge with Reverse GVFs, which are trained via Reverse RL. We demonstrate
empirically the utility of Reverse GVFs in both representation learning and
anomaly detection.
