Zero-Shot Out-of-Distribution Detection Based on the Pre-trained Model
  CLIP
  In an out-of-distribution (OOD) detection problem, samples of known
classes(also called in-distribution classes) are used to train a special
classifier. In testing, the classifier can (1) classify the test samples of
known classes to their respective classes and also (2) detect samples that do
not belong to any of the known classes (i.e., they belong to some unknown or
OOD classes). This paper studies the problem of zero-shot
out-of-distribution(OOD) detection, which still performs the same two tasks in
testing but has no training except using the given known class names. This
paper proposes a novel yet simple method (called ZOC) to solve the problem. ZOC
builds on top of the recent advances in zero-shot classification through
multi-modal representation learning. It first extends the pre-trained
language-vision model CLIP by training a text-based image description generator
on top of CLIP. In testing, it uses the extended model to generate candidate
unknown class names for each test sample and computes a confidence score based
on both the known class names and candidate unknown class names for zero-shot
OOD detection. Experimental results on 5 benchmark datasets for OOD detection
demonstrate that ZOC outperforms the baselines by a large margin.
