Non-decreasing Quantile Function Network with Efficient Exploration for
  Distributional Reinforcement Learning
  Although distributional reinforcement learning (DRL) has been widely examined
in the past few years, there are two open questions people are still trying to
address. One is how to ensure the validity of the learned quantile function,
the other is how to efficiently utilize the distribution information. This
paper attempts to provide some new perspectives to encourage the future
in-depth studies in these two fields. We first propose a non-decreasing
quantile function network (NDQFN) to guarantee the monotonicity of the obtained
quantile estimates and then design a general exploration framework called
distributional prediction error (DPE) for DRL which utilizes the entire
distribution of the quantile function. In this paper, we not only discuss the
theoretical necessity of our method but also show the performance gain it
achieves in practice by comparing with some competitors on Atari 2600 Games
especially in some hard-explored games.
