Shaping Proto-Value Functions via Rewards
  In this paper, we combine task-dependent reward shaping and task-independent
proto-value functions to obtain reward dependent proto-value functions (RPVFs).
In constructing the RPVFs we are making use of the immediate rewards which are
available during the sampling phase but are not used in the PVF construction.
We show via experiments that learning with an RPVF based representation is
better than learning with just reward shaping or PVFs. In particular, when the
state space is symmetrical and the rewards are asymmetrical, the RPVF capture
the asymmetry better than the PVFs.
