Clustered Reinforcement Learning
  Exploration strategy design is one of the challenging problems in
reinforcement learning~(RL), especially when the environment contains a large
state space or sparse rewards. During exploration, the agent tries to discover
novel areas or high reward~(quality) areas. In most existing methods, the
novelty and quality in the neighboring area of the current state are not well
utilized to guide the exploration of the agent. To tackle this problem, we
propose a novel RL framework, called \underline{c}lustered
\underline{r}einforcement \underline{l}earning~(CRL), for efficient exploration
in RL. CRL adopts clustering to divide the collected states into several
clusters, based on which a bonus reward reflecting both novelty and quality in
the neighboring area~(cluster) of the current state is given to the agent.
Experiments on a continuous control task and several \emph{Atari 2600} games
show that CRL can outperform other state-of-the-art methods to achieve the best
performance in most cases.
