On the Minimal Supervision for Training Any Binary Classifier from Only
  Unlabeled Data
  Empirical risk minimization (ERM), with proper loss function and
regularization, is the common practice of supervised classification. In this
paper, we study training arbitrary (from linear to deep) binary classifier from
only unlabeled (U) data by ERM. We prove that it is impossible to estimate the
risk of an arbitrary binary classifier in an unbiased manner given a single set
of U data, but it becomes possible given two sets of U data with different
class priors. These two facts answer a fundamental question---what the minimal
supervision is for training any binary classifier from only U data. Following
these findings, we propose an ERM-based learning method from two sets of U
data, and then prove it is consistent. Experiments demonstrate the proposed
method could train deep models and outperform state-of-the-art methods for
learning from two sets of U data.
