ItNet: iterative neural networks with small graphs for accurate,
  efficient and anytime semantic segmentation
  Deep neural networks have usually to be compressed and accelerated for their
usage in low-power, e.g. mobile, devices. Recently, massively-parallel hardware
accelerators were developed that offer high throughput and low latency at low
power by utilizing in-memory computation. However, to exploit these benefits
the computational graph of a neural network has to fit into the in-computation
memory of these hardware systems that is usually rather limited in size. In
this study, we introduce a class of network models that have a small memory
footprint in terms of their computational graphs. To this end, the graph is
designed to contain loops by iteratively executing a single network building
block. Furthermore, the trade-off between accuracy and latency of these
so-called iterative neural networks is improved by adding multiple intermediate
outputs during both training and inference. We show state-of-the-art results
for semantic segmentation on the CamVid and Cityscapes datasets that are
especially demanding in terms of computational resources. In ablation studies,
the improvement of network training by intermediate network outputs as well as
the trade-off between weight sharing over iterations and the network size are
investigated.
