AttendSeg: A Tiny Attention Condenser Neural Network for Semantic
  Segmentation on the Edge
  In this study, we introduce \textbf{AttendSeg}, a low-precision, highly
compact deep neural network tailored for on-device semantic segmentation.
AttendSeg possesses a self-attention network architecture comprising of
light-weight attention condensers for improved spatial-channel selective
attention at a very low complexity. The unique macro-architecture and
micro-architecture design properties of AttendSeg strike a strong balance
between representational power and efficiency, achieved via a machine-driven
design exploration strategy tailored specifically for the task at hand.
Experimental results demonstrated that the proposed AttendSeg can achieve
segmentation accuracy comparable to much larger deep neural networks with
greater complexity while possessing a significantly lower architecture and
computational complexity (requiring as much as >27x fewer MACs, >72x fewer
parameters, and >288x lower weight memory requirements), making it well-suited
for TinyML applications on the edge.
