The Hitchhiker's Guide to Prior-Shift Adaptation
  In many computer vision classification tasks, class priors at test time often
differ from priors on the training set. In the case of such prior shift,
classifiers must be adapted correspondingly to maintain close to optimal
performance. This paper analyzes methods for adaptation of probabilistic
classifiers to new priors and for estimating new priors on an unlabeled test
set. We propose a novel method to address a known issue of prior estimation
methods based on confusion matrices, where inconsistent estimates of decision
probabilities and confusion matrices lead to negative values in the estimated
priors. Experiments on fine-grained image classification datasets provide
insight into the best practice of prior shift estimation and classifier
adaptation, and show that the proposed method achieves state-of-the-art results
in prior adaptation. Applying the best practice to two tasks with naturally
imbalanced priors, learning from web-crawled images and plant species
classification, increased the recognition accuracy by 1.1% and 3.4%
respectively.
