Convolutional Reservoir Computing for World Models
  Recently, reinforcement learning models have achieved great success,
completing complex tasks such as mastering Go and other games with higher
scores than human players. Many of these models collect considerable data on
the tasks and improve accuracy by extracting visual and time-series features
using convolutional neural networks (CNNs) and recurrent neural networks,
respectively. However, these networks have very high computational costs
because they need to be trained by repeatedly using a large volume of past
playing data. In this study, we propose a novel practical approach called
reinforcement learning with convolutional reservoir computing (RCRC) model. The
RCRC model has several desirable features: 1. it can extract visual and
time-series features very fast because it uses random fixed-weight CNN and the
reservoir computing model; 2. it does not require the training data to be
stored because it extracts features without training and decides action with
evolution strategy. Furthermore, the model achieves state of the art score in
the popular reinforcement learning task. Incredibly, we find the random
weight-fixed simple networks like only one dense layer network can also reach
high score in the RL task.
