Probabilistic Label Trees for Extreme Multi-label Classification
  Extreme multi-label classification (XMLC) is a learning task of tagging
instances with a small subset of relevant labels chosen from an extremely large
pool of possible labels. Problems of this scale can be efficiently handled by
organizing labels as a tree, like in hierarchical softmax used for multi-class
problems. In this paper, we thoroughly investigate probabilistic label trees
(PLTs) which can be treated as a generalization of hierarchical softmax for
multi-label problems. We first introduce the PLT model and discuss training and
inference procedures and their computational costs. Next, we prove the
consistency of PLTs for a wide spectrum of performance metrics. To this end, we
upperbound their regret by a function of surrogate-loss regrets of node
classifiers. Furthermore, we consider a problem of training PLTs in a fully
online setting, without any prior knowledge of training instances, their
features, or labels. In this case, both node classifiers and the tree structure
are trained online. We prove a specific equivalence between the fully online
algorithm and an algorithm with a tree structure given in advance. Finally, we
discuss several implementations of PLTs and introduce a new one, napkinXC,
which we empirically evaluate and compare with state-of-the-art algorithms.
