Out-of-Distribution Detection in Dermatology using Input Perturbation
  and Subset Scanning
  Recent advances in deep learning have led to breakthroughs in the development
of automated skin disease classification. As we observe an increasing interest
in these models in the dermatology space, it is crucial to address aspects such
as the robustness towards input data distribution shifts. Current skin disease
models could make incorrect inferences for test samples from different hardware
devices and clinical settings or unknown disease samples, which are
out-of-distribution (OOD) from the training samples. To this end, we propose a
simple yet effective approach that detect these OOD samples prior to making any
decision. The detection is performed via scanning in the latent space
representation (e.g., activations of the inner layers of any pre-trained skin
disease classifier). The input samples could also perturbed to maximise
divergence of OOD samples. We validate our ODD detection approach in two use
cases: 1) identify samples collected from different protocols, and 2) detect
samples from unknown disease classes. Additionally, we evaluate the performance
of the proposed approach and compare it with other state-of-the-art methods.
Furthermore, data-driven dermatology applications may deepen the disparity in
clinical care across racial and ethnic groups since most datasets are reported
to suffer from bias in skin tone distribution. Therefore, we also evaluate the
fairness of these OOD detection methods across different skin tones. Our
experiments resulted in competitive performance across multiple datasets in
detecting OOD samples, which could be used (in the future) to design more
effective transfer learning techniques prior to inferring on these samples.
