Unravelling Small Sample Size Problems in the Deep Learning World
  The growth and success of deep learning approaches can be attributed to two
major factors: availability of hardware resources and availability of large
number of training samples. For problems with large training databases, deep
learning models have achieved superlative performances. However, there are a
lot of \textit{small sample size or $S^3$} problems for which it is not
feasible to collect large training databases. It has been observed that deep
learning models do not generalize well on $S^3$ problems and specialized
solutions are required. In this paper, we first present a review of deep
learning algorithms for small sample size problems in which the algorithms are
segregated according to the space in which they operate, i.e. input space,
model space, and feature space. Secondly, we present Dynamic Attention Pooling
approach which focuses on extracting global information from the most
discriminative sub-part of the feature map. The performance of the proposed
dynamic attention pooling is analyzed with state-of-the-art ResNet model on
relatively small publicly available datasets such as SVHN, C10, C100, and
TinyImageNet.
