Noise mitigation strategies in physical feedforward neural networks
  Physical neural networks are promising candidates for next generation
artificial intelligence hardware. In such architectures, neurons and
connections are physically realized and do not leverage digital concepts with
their practically infinite signal-to-noise ratio to encode, transduce and
transform information. They therefore are prone to noise with a variety of
statistical and architectural properties, and effective strategies leveraging
network-inherent assets to mitigate noise in an hardware-efficient manner are
important in the pursuit of next generation neural network hardware. Based on
analytical derivations, we here introduce and analyse a variety of different
noise-mitigation approaches. We analytically show that intra-layer connections
in which the connection matrix's squared mean exceeds the mean of its square
fully suppresses uncorrelated noise. We go beyond and develop two synergistic
strategies for noise that is uncorrelated and correlated across populations of
neurons. First, we introduce the concept of ghost neurons, where each group of
neurons perturbed by correlated noise has a negative connection to a single
neuron, yet without receiving any input information. Secondly, we show that
pooling of neuron populations is an efficient approach to suppress uncorrelated
noise. As such, we developed a general noise mitigation strategy leveraging the
statistical properties of the different noise terms most relevant in analogue
hardware. Finally, we demonstrate the effectiveness of this combined approach
for trained neural network classifying the MNIST handwritten digits, for which
we achieve a 4-fold improvement of the output signal-to-noise ratio and
increase the classification accuracy almost to the level of the noise-free
network.
