Deep Fried Convnets
  The fully connected layers of a deep convolutional neural network typically
contain over 90% of the network parameters, and consume the majority of the
memory required to store the network parameters. Reducing the number of
parameters while preserving essentially the same predictive performance is
critically important for operating deep neural networks in memory constrained
environments such as GPUs or embedded devices.
  In this paper we show how kernel methods, in particular a single Fastfood
layer, can be used to replace all fully connected layers in a deep
convolutional neural network. This novel Fastfood layer is also end-to-end
trainable in conjunction with convolutional layers, allowing us to combine them
into a new architecture, named deep fried convolutional networks, which
substantially reduces the memory footprint of convolutional networks trained on
MNIST and ImageNet with no drop in predictive performance.
