Exploration in Deep Reinforcement Learning: A Survey
This paper reviews exploration techniques in deep reinforcement learning.
Exploration techniques are of primary importance when solving sparse reward
problems. In sparse reward problems, the reward is rare, which means that the
agent will not find the reward often by acting randomly. In such a scenario, it
is challenging for reinforcement learning to learn rewards and actions
association. Thus more sophisticated exploration methods need to be devised.
This review provides a comprehensive overview of existing exploration
approaches, which are categorized based on the key contributions as follows
reward novel states, reward diverse behaviours, goal-based methods,
probabilistic methods, imitation-based methods, safe exploration and
random-based methods. Then, the unsolved challenges are discussed to provide
valuable future research directions. Finally, the approaches of different
categories are compared in terms of complexity, computational effort and
overall performance.