Augmenting Supervised Learning by Meta-learning Unsupervised Local Rules
  The brain performs unsupervised learning and (perhaps) simultaneous
supervised learning. This raises the question as to whether a hybrid of
supervised and unsupervised methods will produce better learning. Inspired by
the rich space of Hebbian learning rules, we set out to directly learn the
unsupervised learning rule on local information that best augments a supervised
signal. We present the Hebbian-augmented training algorithm (HAT) for combining
gradient-based learning with an unsupervised rule on pre-synpatic activity,
post-synaptic activities, and current weights. We test HAT's effect on a simple
problem (Fashion-MNIST) and find consistently higher performance than
supervised learning alone. This finding provides empirical evidence that
unsupervised learning on synaptic activities provides a strong signal that can
be used to augment gradient-based methods.
  We further find that the meta-learned update rule is a time-varying function;
thus, it is difficult to pinpoint an interpretable Hebbian update rule that
aids in training. We do find that the meta-learner eventually degenerates into
a non-Hebbian rule that preserves important weights so as not to disturb the
learner's convergence.
