NROWAN-DQN: A Stable Noisy Network with Noise Reduction and Online
  Weight Adjustment for Exploration
  Deep reinforcement learning has been applied more and more widely nowadays,
especially in various complex control tasks. Effective exploration for noisy
networks is one of the most important issues in deep reinforcement learning.
Noisy networks tend to produce stable outputs for agents. However, this
tendency is not always enough to find a stable policy for an agent, which
decreases efficiency and stability during the learning process. Based on
NoisyNets, this paper proposes an algorithm called NROWAN-DQN, i.e., Noise
Reduction and Online Weight Adjustment NoisyNet-DQN. Firstly, we develop a
novel noise reduction method for NoisyNet-DQN to make the agent perform stable
actions. Secondly, we design an online weight adjustment strategy for noise
reduction, which improves stable performance and gets higher scores for the
agent. Finally, we evaluate this algorithm in four standard domains and analyze
properties of hyper-parameters. Our results show that NROWAN-DQN outperforms
prior algorithms in all these domains. In addition, NROWAN-DQN also shows
better stability. The variance of the NROWAN-DQN score is significantly
reduced, especially in some action-sensitive environments. This means that in
some environments where high stability is required, NROWAN-DQN will be more
appropriate than NoisyNets-DQN.
