Proximal Iteration for Deep Reinforcement Learning
  We employ Proximal Iteration for value-function optimization in deep
reinforcement learning. Proximal Iteration is a computationally efficient
technique that enables biasing the optimization procedure towards desirable
solutions. As a concrete application, we endow the objective function of Deep
Q-Network (DQN) and Rainbow agents with a proximal term to ensure robustness in
presence of large noise. The resultant agents, which we call DQN Pro and
Rainbow Pro, exhibit significant improvements over their original counterparts
on the Atari benchmark. Our results accentuate the power of employing sound
optimization techniques for deep reinforcement learning.
