An Efficient Hardware-Oriented Dropout Algorithm
  This paper proposes a hardware-oriented dropout algorithm, which is efficient
for field programmable gate array (FPGA) implementation. In deep neural
networks (DNNs), overfitting occurs when networks are overtrained and adapt too
well to training data. Consequently, they fail in predicting unseen data used
as test data. Dropout is a common technique that is often applied in DNNs to
overcome this problem. In general, implementing such training algorithms of
DNNs in embedded systems is difficult due to power and memory constraints.
Training DNNs is power-, time-, and memory- intensive; however, embedded
systems require low power consumption and real-time processing. An FPGA is
suitable for embedded systems for its parallel processing characteristic and
low operating power; however, due to its limited memory and different
architecture, it is difficult to apply general neural network algorithms.
Therefore, we propose a hardware-oriented dropout algorithm that can
effectively utilize the characteristics of an FPGA with less memory required.
Software program verification demonstrates that the performance of the proposed
method is identical to that of conventional dropout, and hardware synthesis
demonstrates that it results in significant resource reduction.
