An FPGA Accelerated Method for Training Feed-forward Neural Networks
  Using Alternating Direction Method of Multipliers and LSMR
  In this project, we have successfully designed, implemented, deployed and
tested a novel FPGA accelerated algorithm for neural network training. The
algorithm itself was developed in an independent study option. This training
method is based on Alternating Direction Method of Multipliers algorithm, which
has strong parallel characteristics and avoids procedures such as matrix
inversion that are problematic in hardware designs by employing LSMR. As an
intermediate stage, we fully implemented the ADMM-LSMR method in C language for
feed-forward neural networks with a flexible number of layers and hidden size.
We demonstrated that the method can operate with fixed-point arithmetic without
compromising the accuracy. Next, we devised an FPGA accelerated version of the
algorithm using Intel FPGA SDK for OpenCL and performed extensive optimisation
stages followed by successful deployment of the program on an Intel Arria 10 GX
FPGA. The FPGA accelerated program showed up to 6 times speed up comparing to
equivalent CPU implementation while achieving promising accuracy.
