Do We Need Fully Connected Output Layers in Convolutional Networks?
  Traditionally, deep convolutional neural networks consist of a series of
convolutional and pooling layers followed by one or more fully connected (FC)
layers to perform the final classification. While this design has been
successful, for datasets with a large number of categories, the fully connected
layers often account for a large percentage of the network's parameters. For
applications with memory constraints, such as mobile devices and embedded
platforms, this is not ideal. Recently, a family of architectures that involve
replacing the learned fully connected output layer with a fixed layer has been
proposed as a way to achieve better efficiency. In this paper we examine this
idea further and demonstrate that fixed classifiers offer no additional benefit
compared to simply removing the output layer along with its parameters. We
further demonstrate that the typical approach of having a fully connected final
output layer is inefficient in terms of parameter count. We are able to achieve
comparable performance to a traditionally learned fully connected
classification output layer on the ImageNet-1K, CIFAR-100, Stanford Cars-196,
and Oxford Flowers-102 datasets, while not having a fully connected output
layer at all.
