FT-ClipAct: Resilience Analysis of Deep Neural Networks and Improving
  their Fault Tolerance using Clipped Activation
  Deep Neural Networks (DNNs) are widely being adopted for safety-critical
applications, e.g., healthcare and autonomous driving. Inherently, they are
considered to be highly error-tolerant. However, recent studies have shown that
hardware faults that impact the parameters of a DNN (e.g., weights) can have
drastic impacts on its classification accuracy. In this paper, we perform a
comprehensive error resilience analysis of DNNs subjected to hardware faults
(e.g., permanent faults) in the weight memory. The outcome of this analysis is
leveraged to propose a novel error mitigation technique which squashes the
high-intensity faulty activation values to alleviate their impact. We achieve
this by replacing the unbounded activation functions with their clipped
versions. We also present a method to systematically define the clipping values
of the activation functions that result in increased resilience of the networks
against faults. We evaluate our technique on the AlexNet and the VGG-16 DNNs
trained for the CIFAR-10 dataset. The experimental results show that our
mitigation technique significantly improves the resilience of the DNNs to
faults. For example, the proposed technique offers on average 68.92%
improvement in the classification accuracy of resilience-optimized VGG-16 model
at 1e-5 fault rate, when compared to the base network without any fault
mitigation.
