Accelerating Neural ODE Based CNN Models on Low-Cost FPGAs
  ODENet is a deep neural network architecture in which a stacking structure of
ResNet is implemented with an ordinary differential equation (ODE) solver.
ANODE is an extended approach of ODENet for stably high accuracy by introducing
more precise training algorithm using more memory in training phase. It is also
possible to improve the accuracy while keeping the same number of parameters on
resource-limited edge devices. In this paper, using Euler method as an ODE
solver, a part of ODENet and ANODE are implemented as a dedicated logic on a
low-cost FPGA (Field-Programmable Gate Array) board, such as PYNQ-Z2 board. As
ODENet variants, reduced ODENets (rODENets) each of which heavily uses a part
of ODENet layers and reduces/eliminates some layers differently are proposed
and analyzed for low-cost FPGA implementation. In addition, rANODE is proposed
as an ANODE variant in a way similar to the rODENet. They are evaluated in
terms of parameter size, accuracy, execution time, and resource utilization on
the FPGA. The results show that an overall execution time of rODENet and rANODE
variants is improved by up to 2.66 times compared to a pure software execution
while keeping a comparable accuracy to the original ODENet and ANODE.
