Generalized Hindsight for Reinforcement Learning
  One of the key reasons for the high sample complexity in reinforcement
learning (RL) is the inability to transfer knowledge from one task to another.
In standard multi-task RL settings, low-reward data collected while trying to
solve one task provides little to no signal for solving that particular task
and is hence effectively wasted. However, we argue that this data, which is
uninformative for one task, is likely a rich source of information for other
tasks. To leverage this insight and efficiently reuse data, we present
Generalized Hindsight: an approximate inverse reinforcement learning technique
for relabeling behaviors with the right tasks. Intuitively, given a behavior
generated under one task, Generalized Hindsight returns a different task that
the behavior is better suited for. Then, the behavior is relabeled with this
new task before being used by an off-policy RL optimizer. Compared to standard
relabeling techniques, Generalized Hindsight provides a substantially more
efficient reuse of samples, which we empirically demonstrate on a suite of
multi-task navigation and manipulation tasks. Videos and code can be accessed
here: https://sites.google.com/view/generalized-hindsight.
