Emergence of functional and structural properties of the head direction
  system by optimization of recurrent neural networks
  Recent work suggests goal-driven training of neural networks can be used to
model neural activity in the brain. While response properties of neurons in
artificial neural networks bear similarities to those in the brain, the network
architectures are often constrained to be different. Here we ask if a neural
network can recover both neural representations and, if the architecture is
unconstrained and optimized, the anatomical properties of neural circuits. We
demonstrate this in a system where the connectivity and the functional
organization have been characterized, namely, the head direction circuits of
the rodent and fruit fly. We trained recurrent neural networks (RNNs) to
estimate head direction through integration of angular velocity. We found that
the two distinct classes of neurons observed in the head direction system, the
Compass neurons and the Shifter neurons, emerged naturally in artificial neural
networks as a result of training. Furthermore, connectivity analysis and
in-silico neurophysiology revealed structural and mechanistic similarities
between artificial networks and the head direction system. Overall, our results
show that optimization of RNNs in a goal-driven task can recapitulate the
structure and function of biological circuits, suggesting that artificial
neural networks can be used to study the brain at the level of both neural
activity and anatomical organization.
