Learning from Multiple Noisy Partial Labelers
  Programmatic weak supervision creates models without hand-labeled training
data by combining the outputs of heuristic labelers. Existing frameworks make
the restrictive assumption that labelers output a single class label. Enabling
users to create partial labelers that output subsets of possible class labels
would greatly expand the expressivity of programmatic weak supervision. We
introduce this capability by defining a probabilistic generative model that can
estimate the underlying accuracies of multiple noisy partial labelers without
ground truth labels. We show how to scale up learning, for example learning on
100k examples in one minute, a 300x speed up compared to a naive
implementation. We also prove that this class of models is generically
identifiable up to label swapping under mild conditions. We evaluate our
framework on three text classification and six object classification tasks. On
text tasks, adding partial labels increases average accuracy by 8.6 percentage
points. On image tasks, we show that partial labels allow us to approach some
zero-shot object classification problems with programmatic weak supervision by
using class attributes as partial labelers. On these tasks, our framework has
accuracy comparable to recent embedding-based zero-shot learning methods, while
using only pre-trained attribute detectors.
