SIBRE: Self Improvement Based REwards for Adaptive Feedback in
  Reinforcement Learning
  We propose a generic reward shaping approach for improving the rate of
convergence in reinforcement learning (RL), called Self Improvement Based
REwards, or SIBRE. The approach is designed for use in conjunction with any
existing RL algorithm, and consists of rewarding improvement over the agent's
own past performance. We prove that SIBRE converges in expectation under the
same conditions as the original RL algorithm. The reshaped rewards help
discriminate between policies when the original rewards are weakly
discriminated or sparse. Experiments on several well-known benchmark
environments with different RL algorithms show that SIBRE converges to the
optimal policy faster and more stably. We also perform sensitivity analysis
with respect to hyper-parameters, in comparison with baseline RL algorithms.
