Balancing Accuracy and Latency in Multipath Neural Networks
  The growing capacity of neural networks has strongly contributed to their
success at complex machine learning tasks and the computational demand of such
large models has, in turn, stimulated a significant improvement in the hardware
necessary to accelerate their computations. However, models with high latency
aren't suitable for limited-resource environments such as hand-held and IoT
devices. Hence, many deep learning techniques aim to address this problem by
developing models with reasonable accuracy without violating the
limited-resource constraint. In this work, we use a one-shot neural
architecture search model to implicitly evaluate the performance of an
intractable number of multipath neural networks. Combining this architecture
search with a pruning technique and architecture sample evaluation, we can
model the relation between the accuracy and the latency of a spectrum of models
with graded complexity. We show that our method can accurately model the
relative performance between models with different latencies and predict the
performance of unseen models with good precision across different datasets.
