Divergent representations of ethological visual inputs emerge from
  supervised, unsupervised, and reinforcement learning
  Artificial neural systems trained using reinforcement, supervised, and
unsupervised learning all acquire internal representations of high dimensional
input. To what extent these representations depend on the different learning
objectives is largely unknown. Here we compare the representations learned by
eight different convolutional neural networks, each with identical ResNet
architectures and trained on the same family of egocentric images, but embedded
within different learning systems. Specifically, the representations are
trained to guide action in a compound reinforcement learning task; to predict
one or a combination of three task-related targets with supervision; or using
one of three different unsupervised objectives. Using representational
similarity analysis, we find that the network trained with reinforcement
learning differs most from the other networks. Using metrics inspired by the
neuroscience literature, we find that the model trained with reinforcement
learning has a sparse and high-dimensional representation wherein individual
images are represented with very different patterns of neural activity. Further
analysis suggests these representations may arise in order to guide long-term
behavior and goal-seeking in the RL agent. Finally, we compare the
representations learned by the RL agent to neural activity from mouse visual
cortex and find it to perform as well or better than other models. Our results
provide insights into how the properties of neural representations are
influenced by objective functions and can inform transfer learning approaches.
