Strategy and Benchmark for Converting Deep Q-Networks to Event-Driven
  Spiking Neural Networks
  Spiking neural networks (SNNs) have great potential for energy-efficient
implementation of Deep Neural Networks (DNNs) on dedicated neuromorphic
hardware. Recent studies demonstrated competitive performance of SNNs compared
with DNNs on image classification tasks, including CIFAR-10 and ImageNet data.
The present work focuses on using SNNs in combination with deep reinforcement
learning in ATARI games, which involves additional complexity as compared to
image classification. We review the theory of converting DNNs to SNNs and
extending the conversion to Deep Q-Networks (DQNs). We propose a robust
representation of the firing rate to reduce the error during the conversion
process. In addition, we introduce a new metric to evaluate the conversion
process by comparing the decisions made by the DQN and SNN, respectively. We
also analyze how the simulation time and parameter normalization influence the
performance of converted SNNs. We achieve competitive scores on 17
top-performing Atari games. To the best of our knowledge, our work is the first
to achieve state-of-the-art performance on multiple Atari games with SNNs. Our
work serves as a benchmark for the conversion of DQNs to SNNs and paves the way
for further research on solving reinforcement learning tasks with SNNs.
