Sample Efficient Actor-Critic with Experience Replay
  This paper presents an actor-critic deep reinforcement learning agent with experience replay that is stable, sample efficient, and performs remarkably well on challenging environments, including the discrete 57-game Atari domain and several continuous control problems. To achieve this, the paper introduces several innovations, including truncated importance sampling with bias correction, stochastic dueling network architectures, and a new trust region policy optimization method. 
Ziyu Wang, Victor Bapst, Nicolas Heess, Volodymyr Mnih, Remi Munos,   Koray Kavukcuoglu, Nando de Freitas
cs.LG