Similarities between policy gradient methods (PGM) in Reinforcement
  learning (RL) and supervised learning (SL)
  Reinforcement learning (RL) is about sequential decision making and is
traditionally opposed to supervised learning (SL) and unsupervised learning
(USL). In RL, given the current state, the agent makes a decision that may
influence the next state as opposed to SL (and USL) where, the next state
remains the same, regardless of the decisions taken, either in batch or online
learning. Although this difference is fundamental between SL and RL, there are
connections that have been overlooked. In particular, we prove in this paper
that gradient policy method can be cast as a supervised learning problem where
true label are replaced with discounted rewards. We provide a new proof of
policy gradient methods (PGM) that emphasizes the tight link with the cross
entropy and supervised learning. We provide a simple experiment where we
interchange label and pseudo rewards. We conclude that other relationships with
SL could be made if we modify the reward functions wisely.
