Improving Neural Network Generalization by Combining Parallel Circuits
  with Dropout
  In an attempt to solve the lengthy training times of neural networks, we
proposed Parallel Circuits (PCs), a biologically inspired architecture.
Previous work has shown that this approach fails to maintain generalization
performance in spite of achieving sharp speed gains. To address this issue, and
motivated by the way Dropout prevents node co-adaption, in this paper, we
suggest an improvement by extending Dropout to the PC architecture. The paper
provides multiple insights into this combination, including a variety of fusion
approaches. Experiments show promising results in which improved error rates
are achieved in most cases, whilst maintaining the speed advantage of the PC
approach.
