Gated Transfer Network for Transfer Learning
  Deep neural networks have led to a series of breakthroughs in computer vision
given sufficient annotated training datasets. For novel tasks with limited
labeled data, the prevalent approach is to transfer the knowledge learned in
the pre-trained models to the new tasks by fine-tuning. Classic model
fine-tuning utilizes the fact that well trained neural networks appear to learn
cross domain features. These features are treated equally during transfer
learning. In this paper, we explore the impact of feature selection in model
fine-tuning by introducing a transfer module, which assigns weights to features
extracted from pre-trained models. The proposed transfer module proves the
importance of feature selection for transferring models from source to target
domains. It is shown to significantly improve upon fine-tuning results with
only marginal extra computational cost. We also incorporate an auxiliary
classifier as an extra regularizer to avoid over-fitting. Finally, we build a
Gated Transfer Network (GTN) based on our transfer module and achieve
state-of-the-art results on six different tasks.
