Hebbian Semi-Supervised Learning in a Sample Efficiency Setting
  We propose to address the issue of sample efficiency, in Deep Convolutional
Neural Networks (DCNN), with a semi-supervised training strategy that combines
Hebbian learning with gradient descent: all internal layers (both convolutional
and fully connected) are pre-trained using an unsupervised approach based on
Hebbian learning, and the last fully connected layer (the classification layer)
is trained using Stochastic Gradient Descent (SGD). In fact, as Hebbian
learning is an unsupervised learning method, its potential lies in the
possibility of training the internal layers of a DCNN without labels. Only the
final fully connected layer has to be trained with labeled examples.
  We performed experiments on various object recognition datasets, in different
regimes of sample efficiency, comparing our semi-supervised (Hebbian for
internal layers + SGD for the final fully connected layer) approach with
end-to-end supervised backprop training, and with semi-supervised learning
based on Variational Auto-Encoder (VAE). The results show that, in regimes
where the number of available labeled samples is low, our semi-supervised
approach outperforms the other approaches in almost all the cases.
