Understanding Robustness in Teacher-Student Setting: A New Perspective
  Adversarial examples have appeared as a ubiquitous property of machine
learning models where bounded adversarial perturbation could mislead the models
to make arbitrarily incorrect predictions. Such examples provide a way to
assess the robustness of machine learning models as well as a proxy for
understanding the model training process. Extensive studies try to explain the
existence of adversarial examples and provide ways to improve model robustness
(e.g. adversarial training). While they mostly focus on models trained on
datasets with predefined labels, we leverage the teacher-student framework and
assume a teacher model, or oracle, to provide the labels for given instances.
We extend Tian (2019) in the case of low-rank input data and show that student
specialization (trained student neuron is highly correlated with certain
teacher neuron at the same layer) still happens within the input subspace, but
the teacher and student nodes could differ wildly out of the data subspace,
which we conjecture leads to adversarial examples. Extensive experiments show
that student specialization correlates strongly with model robustness in
different scenarios, including student trained via standard training,
adversarial training, confidence-calibrated adversarial training, and training
with robust feature dataset. Our studies could shed light on the future
exploration about adversarial examples, and enhancing model robustness via
principled data augmentation.
