Towards Enhancing Fault Tolerance in Neural Networks
  Deep Learning Accelerators are prone to faults which manifest in the form of
errors in Neural Networks. Fault Tolerance in Neural Networks is crucial in
real-time safety critical applications requiring computation for long
durations. Neural Networks with high regularisation exhibit superior fault
tolerance, however, at the cost of classification accuracy. In the view of
difference in functionality, a Neural Network is modelled as two separate
networks, i.e, the Feature Extractor with unsupervised learning objective and
the Classifier with a supervised learning objective. Traditional approaches of
training the entire network using a single supervised learning objective is
insufficient to achieve the objectives of the individual components optimally.
In this work, a novel multi-criteria objective function, combining unsupervised
training of the Feature Extractor followed by supervised tuning with Classifier
Network is proposed. The unsupervised training solves two games simultaneously
in the presence of adversary neural networks with conflicting objectives to the
Feature Extractor. The first game minimises the loss in reconstructing the
input image for indistinguishability given the features from the Extractor, in
the presence of a generative decoder. The second game solves a minimax
constraint optimisation for distributional smoothening of feature space to
match a prior distribution, in the presence of a Discriminator network. The
resultant strongly regularised Feature Extractor is combined with the
Classifier Network for supervised fine-tuning. The proposed Adversarial Fault
Tolerant Neural Network Training is scalable to large networks and is
independent of the architecture. The evaluation on benchmarking datasets:
FashionMNIST and CIFAR10, indicates that the resultant networks have high
accuracy with superior tolerance to stuck at "0" faults compared to widely used
regularisers.
