Surprisal-Triggered Conditional Computation with Neural Networks
  Autoregressive neural network models have been used successfully for sequence
generation, feature extraction, and hypothesis scoring. This paper presents yet
another use for these models: allocating more computation to more difficult
inputs. In our model, an autoregressive model is used both to extract features
and to predict observations in a stream of input observations. The surprisal of
the input, measured as the negative log-likelihood of the current observation
according to the autoregressive model, is used as a measure of input
difficulty. This in turn determines whether a small, fast network, or a big,
slow network, is used. Experiments on two speech recognition tasks show that
our model can match the performance of a baseline in which the big network is
always used with 15% fewer FLOPs.
