Improving image classifiers for small datasets by learning rate
  adaptations
  Our paper introduces an efficient combination of established techniques to
improve classifier performance, in terms of accuracy and training time. We
achieve two-fold to ten-fold speedup in nearing state of the art accuracy, over
different model architectures, by dynamically tuning the learning rate. We find
it especially beneficial in the case of a small dataset, where reliability of
machine reasoning is lower. We validate our approach by comparing our method
versus vanilla training on CIFAR-10. We also demonstrate its practical
viability by implementing on an unbalanced corpus of diagnostic images.
