ULF: Unsupervised Labeling Function Correction using Cross-Validation
  for Weak Supervision
  A way to overcome expensive and time-consuming manual data labeling is weak
supervision - automatic annotation of data samples via a predefined set of
labeling functions (LFs), rule-based mechanisms that generate potentially
erroneous labels. In this work, we investigate noise reduction techniques for
weak supervision based on the principle of k-fold cross-validation. In
particular, we extend two frameworks for detecting the erroneous samples in
manually annotated data to the weakly supervised setting. Our methods profit
from leveraging the information about matching LFs and detect noisy samples
more accurately. We also introduce a new algorithm for denoising the weakly
annotated data called ULF, that refines the allocation of LFs to classes by
estimating the reliable LFs-to-classes joint matrix. Evaluation on several
datasets shows that ULF successfully improves weakly supervised learning
without using any manually labeled data.
