Dynamically Throttleable Neural Networks (TNN)
  Conditional computation for Deep Neural Networks (DNNs) reduce overall
computational load and improve model accuracy by running a subset of the
network. In this work, we present a runtime throttleable neural network (TNN)
that can adaptively self-regulate its own performance target and computing
resources. We designed TNN with several properties that enable more flexibility
for dynamic execution based on runtime context. TNNs are defined as
throttleable modules gated with a separately trained controller that generates
a single utilization control parameter. We validate our proposal on a number of
experiments, including Convolution Neural Networks (CNNs such as VGG, ResNet,
ResNeXt, DenseNet) using CiFAR-10 and ImageNet dataset, for object
classification and recognition tasks. We also demonstrate the effectiveness of
dynamic TNN execution on a 3D Convolustion Network (C3D) for a hand gesture
task. Results show that TNN can maintain peak accuracy performance compared to
vanilla solutions, while providing a graceful reduction in computational
requirement, down to 74% reduction in latency and 52% energy savings.
