Learning Curves for Analysis of Deep Networks
  Learning curves model a classifier's test error as a function of the number
of training samples. Prior works show that learning curves can be used to
select model parameters and extrapolate performance. We investigate how to use
learning curves to evaluate design choices, such as pretraining, architecture,
and data augmentation. We propose a method to robustly estimate learning
curves, abstract their parameters into error and data-reliance, and evaluate
the effectiveness of different parameterizations. Our experiments exemplify use
of learning curves for analysis and yield several interesting observations.
