Aggressive Sampling for Multi-class to Binary Reduction with
  Applications to Text Classification
  We address the problem of multi-class classification in the case where the
number of classes is very large. We propose a double sampling strategy on top
of a multi-class to binary reduction strategy, which transforms the original
multi-class problem into a binary classification problem over pairs of
examples. The aim of the sampling strategy is to overcome the curse of
long-tailed class distributions exhibited in majority of large-scale
multi-class classification problems and to reduce the number of pairs of
examples in the expanded data. We show that this strategy does not alter the
consistency of the empirical risk minimization principle defined over the
double sample reduction. Experiments are carried out on DMOZ and Wikipedia
collections with 10,000 to 100,000 classes where we show the efficiency of the
proposed approach in terms of training and prediction time, memory consumption,
and predictive performance with respect to state-of-the-art approaches.
