Model Reprogramming: Resource-Efficient Cross-Domain Machine Learning
  In data-rich domains such as vision, language, and speech, deep learning
prevails to deliver high-performance task-specific models and can even learn
general task-agnostic representations for efficient finetuning to downstream
tasks. However, deep learning in resource-limited domains still faces the
following challenges including (i) limited data, (ii) constrained model
development cost, and (iii) lack of adequate pre-trained models for effective
finetuning. This paper introduces a new technique called model reprogramming to
bridge this gap. Model reprogramming enables resource-efficient cross-domain
machine learning by repurposing and reusing a well-developed pre-trained model
from a source domain to solve tasks in a target domain without model
finetuning, where the source and target domains can be vastly different. In
many applications, model reprogramming outperforms transfer learning and
training from scratch. This paper elucidates the methodology of model
reprogramming, summarizes existing use cases, provides a theoretical
explanation on the success of model reprogramming, and concludes with a
discussion on open-ended research questions and opportunities. A list of model
reprogramming studies is actively maintained and updated at
https://github.com/IBM/model-reprogramming.
