Neural networks on microcontrollers: saving memory at inference via
  operator reordering
  Designing deep learning models for highly-constrained hardware would allow
imbuing many edge devices with intelligence. Microcontrollers (MCUs) are an
attractive platform for building smart devices due to their low cost, wide
availability, and modest power usage. However, they lack the computational
resources to run neural networks as straightforwardly as mobile or server
platforms, which necessitates changes to the network architecture and the
inference software. In this work, we discuss the deployment and memory concerns
of neural networks on MCUs and present a way of saving memory by changing the
execution order of the network's operators, which is orthogonal to other
compression methods. We publish a tool for reordering operators of TensorFlow
Lite models and demonstrate its utility by sufficiently reducing the memory
footprint of a CNN to deploy it on an MCU with 512KB SRAM.
