Faster learning of deep stacked autoencoders on multi-core systems using
  synchronized layer-wise pre-training
  Deep neural networks are capable of modelling highly non-linear functions by
capturing different levels of abstraction of data hierarchically. While
training deep networks, first the system is initialized near a good optimum by
greedy layer-wise unsupervised pre-training. However, with burgeoning data and
increasing dimensions of the architecture, the time complexity of this approach
becomes enormous. Also, greedy pre-training of the layers often turns
detrimental by over-training a layer causing it to lose harmony with the rest
of the network. In this paper a synchronized parallel algorithm for
pre-training deep networks on multi-core machines has been proposed. Different
layers are trained by parallel threads running on different cores with regular
synchronization. Thus the pre-training process becomes faster and chances of
over-training are reduced. This is experimentally validated using a stacked
autoencoder for dimensionality reduction of MNIST handwritten digit database.
The proposed algorithm achieved 26\% speed-up compared to greedy layer-wise
pre-training for achieving the same reconstruction accuracy substantiating its
potential as an alternative.
