An Out-of-the-box Full-network Embedding for Convolutional Neural
  Networks
  Transfer learning for feature extraction can be used to exploit deep
representations in contexts where there is very few training data, where there
are limited computational resources, or when tuning the hyper-parameters needed
for training is not an option. While previous contributions to feature
extraction propose embeddings based on a single layer of the network, in this
paper we propose a full-network embedding which successfully integrates
convolutional and fully connected features, coming from all layers of a deep
convolutional neural network. To do so, the embedding normalizes features in
the context of the problem, and discretizes their values to reduce noise and
regularize the embedding space. Significantly, this also reduces the
computational cost of processing the resultant representations. The proposed
method is shown to outperform single layer embeddings on several image
classification tasks, while also being more robust to the choice of the
pre-trained model used for obtaining the initial features. The performance gap
in classification accuracy between thoroughly tuned solutions and the
full-network embedding is also reduced, which makes of the proposed approach a
competitive solution for a large set of applications.
