Recurrent Feedback Improves Feedforward Representations in Deep Neural
  Networks
  The abundant recurrent horizontal and feedback connections in the primate
visual cortex are thought to play an important role in bringing global and
semantic contextual information to early visual areas during perceptual
inference, helping to resolve local ambiguity and fill in missing details. In
this study, we find that introducing feedback loops and horizontal recurrent
connections to a deep convolution neural network (VGG16) allows the network to
become more robust against noise and occlusion during inference, even in the
initial feedforward pass. This suggests that recurrent feedback and contextual
modulation transform the feedforward representations of the network in a
meaningful and interesting way. We study the population codes of neurons in the
network, before and after learning with feedback, and find that learning with
feedback yielded an increase in discriminability (measured by d-prime) between
the different object classes in the population codes of the neurons in the
feedforward path, even at the earliest layer that receives feedback. We find
that recurrent feedback, by injecting top-down semantic meaning to the
population activities, helps the network learn better feedforward paths to
robustly map noisy image patches to the latent representations corresponding to
important visual concepts of each object class, resulting in greater robustness
of the network against noises and occlusion as well as better fine-grained
recognition.
