Reinforcement Learning with Convolutional Reservoir Computing
  Recently, reinforcement learning models have achieved great success,
mastering complex tasks such as Go and other games with higher scores than
human players. Many of these models store considerable data on the tasks and
achieve high performance by extracting visual and time-series features using
convolutional neural networks (CNNs) and recurrent neural networks,
respectively. However, these networks have very high computational costs
because they need to be trained by repeatedly using the stored data. In this
study, we propose a novel practical approach called reinforcement learning with
convolutional reservoir computing (RCRC) model. The RCRC model uses a fixed
random-weight CNN and a reservoir computing model to extract visual and
time-series features. Using these extracted features, it decides actions with
an evolution strategy method. Thereby, the RCRC model has several desirable
features: (1) there is no need to train the feature extractor, (2) there is no
need to store training data, (3) it can take a wide range of actions, and (4)
there is only a single task-dependent weight parameter to be trained.
Furthermore, we show the RCRC model can solve multiple reinforcement learning
tasks with a completely identical feature extractor.
