Teacher-Student Domain Adaptation for Biosensor Models
  We present an approach to domain adaptation, addressing the case where data
from the source domain is abundant, labelled data from the target domain is
limited or non-existent, and a small amount of paired source-target data is
available. The method is designed for developing deep learning models that
detect the presence of medical conditions based on data from consumer-grade
portable biosensors. It addresses some of the key problems in this area,
namely, the difficulty of acquiring large quantities of clinically labelled
data from the biosensor, and the noise and ambiguity that can affect the
clinical labels. The idea is to pre-train an expressive model on a large
dataset of labelled recordings from a sensor modality for which data is
abundant, and then to adapt the model's lower layers so that its predictions on
the target modality are similar to the original model's on paired examples from
the source modality. We show that the pre-trained model's predictions provide a
substantially better learning signal than the clinician-provided labels, and
that this teacher-student technique significantly outperforms both a naive
application of supervised deep learning and a label-supervised version of
domain adaptation on a synthetic dataset and in a real-world case study on
sleep apnea. By reducing the volume of data required and obviating the need for
labels, our approach should reduce the cost associated with developing
high-performance deep learning models for biosensors.
