At Human Speed: Deep Reinforcement Learning with Action Delay
  There has been a recent explosion in the capabilities of game-playing
artificial intelligence. Many classes of tasks, from video games to motor
control to board games, are now solvable by fairly generic algorithms, based on
deep learning and reinforcement learning, that learn to play from experience
with minimal prior knowledge. However, these machines often do not win through
intelligence alone -- they possess vastly superior speed and precision,
allowing them to act in ways a human never could. To level the playing field,
we restrict the machine's reaction time to a human level, and find that
standard deep reinforcement learning methods quickly drop in performance. We
propose a solution to the action delay problem inspired by human perception --
to endow agents with a neural predictive model of the environment which
"undoes" the delay inherent in their environment -- and demonstrate its
efficacy against professional players in Super Smash Bros. Melee, a popular
console fighting game.
