Deep Learning Training with Simulated Approximate Multipliers
  This paper presents by simulation how approximate multipliers can be utilized
to enhance the training performance of convolutional neural networks (CNNs).
Approximate multipliers have significantly better performance in terms of
speed, power, and area compared to exact multipliers. However, approximate
multipliers have an inaccuracy which is defined in terms of the Mean Relative
Error (MRE). To assess the applicability of approximate multipliers in
enhancing CNN training performance, a simulation for the impact of approximate
multipliers error on CNN training is presented. The paper demonstrates that
using approximate multipliers for CNN training can significantly enhance the
performance in terms of speed, power, and area at the cost of a small negative
impact on the achieved accuracy. Additionally, the paper proposes a hybrid
training method which mitigates this negative impact on the accuracy. Using the
proposed hybrid method, the training can start using approximate multipliers
then switches to exact multipliers for the last few epochs. Using this method,
the performance benefits of approximate multipliers in terms of speed, power,
and area can be attained for a large portion of the training stage. On the
other hand, the negative impact on the accuracy is diminished by using the
exact multipliers for the last epochs of training.
