Dynamic Neural Network Architectural and Topological Adaptation and
  Related Methods -- A Survey
  Training and inference in deep neural networks (DNNs) has, due to a steady
increase in architectural complexity and data set size, lead to the development
of strategies for reducing time and space requirements of DNN training and
inference, which is of particular importance in scenarios where training takes
place in resource constrained computation environments or inference is part of
a time critical application. In this survey, we aim to provide a general
overview and categorization of state-of-the-art (SOTA) of techniques to reduced
DNN training and inference time and space complexities with a particular focus
on architectural adaptions.
