Sample-based Regularization: A Transfer Learning Strategy Toward Better
  Generalization
  Training a deep neural network with a small amount of data is a challenging
problem as it is vulnerable to overfitting. However, one of the practical
difficulties that we often face is to collect many samples. Transfer learning
is a cost-effective solution to this problem. By using the source model trained
with a large-scale dataset, the target model can alleviate the overfitting
originated from the lack of training data. Resorting to the ability of
generalization of the source model, several methods proposed to use the source
knowledge during the whole training procedure. However, this is likely to
restrict the potential of the target model and some transferred knowledge from
the source can interfere with the training procedure. For improving the
generalization performance of the target model with a few training samples, we
proposed a regularization method called sample-based regularization (SBR),
which does not rely on the source's knowledge during training. With SBR, we
suggested a new training framework for transfer learning. Experimental results
showed that our framework outperformed existing methods in various
configurations.
