Improving Memory Utilization in Convolutional Neural Network
  Accelerators
  While the accuracy of convolutional neural networks has achieved vast
improvements by introducing larger and deeper network architectures, also the
memory footprint for storing their parameters and activations has increased.
This trend especially challenges power- and resource-limited accelerator
designs, which are often restricted to store all network data in on-chip memory
to avoid interfacing energy-hungry external memories. Maximizing the network
size that fits on a given accelerator thus requires to maximize its memory
utilization. While the traditionally used ping-pong buffering technique is
mapping subsequent activation layers to disjunctive memory regions, we propose
a mapping method that allows these regions to overlap and thus utilize the
memory more efficiently. This work presents the mathematical model to compute
the maximum activations memory overlap and thus the lower bound of on-chip
memory needed to perform layer-by-layer processing of convolutional neural
networks on memory-limited accelerators. Our experiments with various
real-world object detector networks show that the proposed mapping technique
can decrease the activations memory by up to 32.9%, reducing the overall memory
for the entire network by up to 23.9% compared to traditional ping-pong
buffering. For higher resolution de-noising networks, we achieve activation
memory savings of 48.8%. Additionally, we implement a face detector network on
an FPGA-based camera to validate these memory savings on a complete end-to-end
system.
