Effects of sparse rewards of different magnitudes in the speed of
  learning of model-based actor critic methods
  Actor critic methods with sparse rewards in model-based deep reinforcement
learning typically require a deterministic binary reward function that reflects
only two possible outcomes: if, for each step, the goal has been achieved or
not. Our hypothesis is that we can influence an agent to learn faster by
applying an external environmental pressure during training, which adversely
impacts its ability to get higher rewards. As such, we deviate from the
classical paradigm of sparse rewards and add a uniformly sampled reward value
to the baseline reward to show that (1) sample efficiency of the training
process can be correlated to the adversity experienced during training, (2) it
is possible to achieve higher performance in less time and with less resources,
(3) we can reduce the performance variability experienced seed over seed, (4)
there is a maximum point after which more pressure will not generate better
results, and (5) that random positive incentives have an adverse effect when
using a negative reward strategy, making an agent under those conditions learn
poorly and more slowly. These results have been shown to be valid for Deep
Deterministic Policy Gradients using Hindsight Experience Replay in a well
known Mujoco environment, but we argue that they could be generalized to other
methods and environments as well.
