Neuronal Synchrony in Complex-Valued Deep Networks
  Deep learning has recently led to great successes in tasks such as image
recognition (e.g Krizhevsky et al., 2012). However, deep networks are still
outmatched by the power and versatility of the brain, perhaps in part due to
the richer neuronal computations available to cortical circuits. The challenge
is to identify which neuronal mechanisms are relevant, and to find suitable
abstractions to model them. Here, we show how aspects of spike timing, long
hypothesized to play a crucial role in cortical information processing, could
be incorporated into deep networks to build richer, versatile representations.
  We introduce a neural network formulation based on complex-valued neuronal
units that is not only biologically meaningful but also amenable to a variety
of deep learning frameworks. Here, units are attributed both a firing rate and
a phase, the latter indicating properties of spike timing. We show how this
formulation qualitatively captures several aspects thought to be related to
neuronal synchrony, including gating of information processing and dynamic
binding of distributed object representations. Focusing on the latter, we
demonstrate the potential of the approach in several simple experiments. Thus,
neuronal synchrony could be a flexible mechanism that fulfills multiple
functional roles in deep networks.
