Contrastive Similarity Matching for Supervised Learning
  We propose a novel biologically-plausible solution to the credit assignment
problem motivated by observations in the ventral visual pathway and trained
deep neural networks. In both, representations of objects in the same category
become progressively more similar, while objects belonging to different
categories become less similar. We use this observation to motivate a
layer-specific learning goal in a deep network: each layer aims to learn a
representational similarity matrix that interpolates between previous and later
layers. We formulate this idea using a contrastive similarity matching
objective function and derive from it deep neural networks with feedforward,
lateral, and feedback connections, and neurons that exhibit
biologically-plausible Hebbian and anti-Hebbian plasticity. Contrastive
similarity matching can be interpreted as an energy-based learning algorithm,
but with significant differences from others in how a contrastive function is
constructed.
