AutoFCL: Automatically Tuning Fully Connected Layers for Handling Small
  Dataset
  Deep Convolutional Neural Networks (CNN) have evolved as popular machine
learning models for image classification during the past few years, due to
their ability to learn the problem-specific features directly from the input
images. The success of deep learning models solicits architecture engineering
rather than hand-engineering the features. However, designing state-of-the-art
CNN for a given task remains a non-trivial and challenging task, especially
when training data size is less. To address this phenomena, transfer learning
has been used as a popularly adopted technique. While transferring the learned
knowledge from one task to another, fine-tuning with the target-dependent Fully
Connected (FC) layers generally produces better results over the target task.
In this paper, the proposed AutoFCL model attempts to learn the structure of FC
layers of a CNN automatically using Bayesian optimization. To evaluate the
performance of the proposed AutoFCL, we utilize five pre-trained CNN models
such as VGG-16, ResNet, DenseNet, MobileNet, and NASNetMobile. The experiments
are conducted on three benchmark datasets, namely CalTech-101, Oxford-102
Flowers, and UC Merced Land Use datasets. Fine-tuning the newly learned
(target-dependent) FC layers leads to state-of-the-art performance, according
to the experiments carried out in this research. The proposed AutoFCL method
outperforms the existing methods over CalTech-101 and Oxford-102 Flowers
datasets by achieving the accuracy of 94.38% and 98.89%, respectively. However,
our method achieves comparable performance on the UC Merced Land Use dataset
with 96.83% accuracy. The source codes of this research are available at
https://github.com/shabbeersh/AutoFCL.
