AI can evolve without labels: self-evolving vision transformer for chest
  X-ray diagnosis through knowledge distillation
  Although deep learning-based computer-aided diagnosis systems have recently
achieved expert-level performance, developing a robust deep learning model
requires large, high-quality data with manual annotation, which is expensive to
obtain. This situation poses the problem that the chest x-rays collected
annually in hospitals cannot be used due to the lack of manual labeling by
experts, especially in deprived areas. To address this, here we present a novel
deep learning framework that uses knowledge distillation through
self-supervised learning and self-training, which shows that the performance of
the original model trained with a small number of labels can be gradually
improved with more unlabeled data. Experimental results show that the proposed
framework maintains impressive robustness against a real-world environment and
has general applicability to several diagnostic tasks such as tuberculosis,
pneumothorax, and COVID-19. Notably, we demonstrated that our model performs
even better than those trained with the same amount of labeled data. The
proposed framework has a great potential for medical imaging, where plenty of
data is accumulated every year, but ground truth annotations are expensive to
obtain.
