FitAct: Error Resilient Deep Neural Networks via Fine-Grained
  Post-Trainable Activation Functions
  Deep neural networks (DNNs) are increasingly being deployed in
safety-critical systems such as personal healthcare devices and self-driving
cars. In such DNN-based systems, error resilience is a top priority since
faults in DNN inference could lead to mispredictions and safety hazards. For
latency-critical DNN inference on resource-constrained edge devices, it is
nontrivial to apply conventional redundancy-based fault tolerance techniques.
In this paper, we propose FitAct, a low-cost approach to enhance the error
resilience of DNNs by deploying fine-grained post-trainable activation
functions. The main idea is to precisely bound the activation value of each
individual neuron via neuron-wise bounded activation functions so that it could
prevent fault propagation in the network. To avoid complex DNN model
re-training, we propose to decouple the accuracy training and resilience
training and develop a lightweight post-training phase to learn these
activation functions with precise bound values. Experimental results on widely
used DNN models such as AlexNet, VGG16, and ResNet50 demonstrate that FitAct
outperforms state-of-the-art studies such as Clip-Act and Ranger in enhancing
the DNN error resilience for a wide range of fault rates while adding
manageable runtime and memory space overheads.
