Estimating the Accuracies of Multiple Classifiers Without Labeled Data
  In various situations one is given only the predictions of multiple
classifiers over a large unlabeled test data. This scenario raises the
following questions: Without any labeled data and without any a-priori
knowledge about the reliability of these different classifiers, is it possible
to consistently and computationally efficiently estimate their accuracies?
Furthermore, also in a completely unsupervised manner, can one construct a more
accurate unsupervised ensemble classifier? In this paper, focusing on the
binary case, we present simple, computationally efficient algorithms to solve
these questions. Furthermore, under standard classifier independence
assumptions, we prove our methods are consistent and study their asymptotic
error. Our approach is spectral, based on the fact that the off-diagonal
entries of the classifiers' covariance matrix and 3-d tensor are rank-one. We
illustrate the competitive performance of our algorithms via extensive
experiments on both artificial and real datasets.
