Dynamic Sparse Training for Deep Reinforcement Learning
  Deep reinforcement learning (DRL) agents are trained through trial-and-error
interactions with the environment. This leads to a long training time for dense
neural networks to achieve good performance. Hence, prohibitive computation and
memory resources are consumed. Recently, learning efficient DRL agents has
received increasing attention. Yet, current methods focus on accelerating
inference time. In this paper, we introduce for the first time a dynamic sparse
training approach for deep reinforcement learning to accelerate the training
process. The proposed approach trains a sparse neural network from scratch and
dynamically adapts its topology to the changing data distribution during
training. Experiments on continuous control tasks show that our dynamic sparse
agents achieve higher performance than the equivalent dense methods, reduce the
parameter count and floating-point operations (FLOPs) by 50%, and have a faster
learning speed that enables reaching the performance of dense agents with
40-50% reduction in the training steps.
