Optimizing Bayesian Recurrent Neural Networks on an FPGA-based
  Accelerator
  Neural networks have demonstrated their outstanding performance in a wide
range of tasks. Specifically recurrent architectures based on long-short term
memory (LSTM) cells have manifested excellent capability to model time
dependencies in real-world data. However, standard recurrent architectures
cannot estimate their uncertainty which is essential for safety-critical
applications such as in medicine. In contrast, Bayesian recurrent neural
networks (RNNs) are able to provide uncertainty estimation with improved
accuracy. Nonetheless, Bayesian RNNs are computationally and memory demanding,
which limits their practicality despite their advantages. To address this
issue, we propose an FPGA-based hardware design to accelerate Bayesian
LSTM-based RNNs. To further improve the overall algorithmic-hardware
performance, a co-design framework is proposed to explore the most fitting
algorithmic-hardware configurations for Bayesian RNNs. We conduct extensive
experiments on healthcare applications to demonstrate the improvement of our
design and the effectiveness of our framework. Compared with GPU
implementation, our FPGA-based design can achieve up to 10 times speedup with
nearly 106 times higher energy efficiency. To the best of our knowledge, this
is the first work targeting acceleration of Bayesian RNNs on FPGAs.
