Active Learning for Open-set Annotation
  Existing active learning studies typically work in the closed-set setting by
assuming that all data examples to be labeled are drawn from known classes.
However, in real annotation tasks, the unlabeled data usually contains a large
amount of examples from unknown classes, resulting in the failure of most
active learning methods. To tackle this open-set annotation (OSA) problem, we
propose a new active learning framework called LfOSA, which boosts the
classification performance with an effective sampling strategy to precisely
detect examples from known classes for annotation. The LfOSA framework
introduces an auxiliary network to model the per-example max activation value
(MAV) distribution with a Gaussian Mixture Model, which can dynamically select
the examples with highest probability from known classes in the unlabeled set.
Moreover, by reducing the temperature $T$ of the loss function, the detection
model will be further optimized by exploiting both known and unknown
supervision. The experimental results show that the proposed method can
significantly improve the selection quality of known classes, and achieve
higher classification accuracy with lower annotation cost than state-of-the-art
active learning methods. To the best of our knowledge, this is the first work
of active learning for open-set annotation.
