Improvements on Hindsight Learning
  Sparse reward problems are one of the biggest challenges in Reinforcement
Learning. Goal-directed tasks are one such sparse reward problems where a
reward signal is received only when the goal is reached. One promising way to
train an agent to perform goal-directed tasks is to use Hindsight Learning
approaches. In these approaches, even when an agent fails to reach the desired
goal, the agent learns to reach the goal it achieved instead. Doing this over
multiple trajectories while generalizing the policy learned from the achieved
goals, the agent learns a goal conditioned policy to reach any goal. One such
approach is Hindsight Experience replay which uses an off-policy Reinforcement
Learning algorithm to learn a goal conditioned policy. In this approach, a
replay of the past transitions happens in a uniformly random fashion. Another
approach is to use a Hindsight version of the policy gradients to directly
learn a policy. In this work, we discuss different ways to replay past
transitions to improve learning in hindsight experience replay focusing on
prioritized variants in particular. Also, we implement the Hindsight Policy
gradient methods to robotic tasks.
