Training Progressively Binarizing Deep Networks Using FPGAs
  While hardware implementations of inference routines for Binarized Neural
Networks (BNNs) are plentiful, current realizations of efficient BNN hardware
training accelerators, suitable for Internet of Things (IoT) edge devices,
leave much to be desired. Conventional BNN hardware training accelerators
perform forward and backward propagations with parameters adopting binary
representations, and optimization using parameters adopting floating or
fixed-point real-valued representations--requiring two distinct sets of network
parameters. In this paper, we propose a hardware-friendly training method that,
contrary to conventional methods, progressively binarizes a singular set of
fixed-point network parameters, yielding notable reductions in power and
resource utilizations. We use the Intel FPGA SDK for OpenCL development
environment to train our progressively binarizing DNNs on an OpenVINO FPGA. We
benchmark our training approach on both GPUs and FPGAs using CIFAR-10 and
compare it to conventional BNNs.
