Transfer Learning without Knowing: Reprogramming Black-box Machine
  Learning Models with Scarce Data and Limited Resources
  Current transfer learning methods are mainly based on finetuning a pretrained
model with target-domain data. Motivated by the techniques from adversarial
machine learning (ML) that are capable of manipulating the model prediction via
data perturbations, in this paper we propose a novel approach, black-box
adversarial reprogramming (BAR), that repurposes a well-trained black-box ML
model (e.g., a prediction API or a proprietary software) for solving different
ML tasks, especially in the scenario with scarce data and constrained
resources. The rationale lies in exploiting high-performance but unknown ML
models to gain learning capability for transfer learning. Using zeroth order
optimization and multi-label mapping techniques, BAR can reprogram a black-box
ML model solely based on its input-output responses without knowing the model
architecture or changing any parameter. More importantly, in the limited
medical data setting, on autism spectrum disorder classification, diabetic
retinopathy detection, and melanoma detection tasks, BAR outperforms
state-of-the-art methods and yields comparable performance to the vanilla
adversarial reprogramming method requiring complete knowledge of the target ML
model. BAR also outperforms baseline transfer learning approaches by a
significant margin, demonstrating cost-effective means and new insights for
transfer learning.
