Efficient Memory Management for Deep Neural Net Inference
  While deep neural net inference was considered a task for servers only,
latest advances in technology allow the task of inference to be moved to mobile
and embedded devices, desired for various reasons ranging from latency to
privacy. These devices are not only limited by their compute power and battery,
but also by their inferior physical memory and cache, and thus, an efficient
memory manager becomes a crucial component for deep neural net inference at the
edge. We explore various strategies to smartly share memory buffers among
intermediate tensors in deep neural nets. Employing these can result in up to
11% smaller memory footprint than the state of the art.
