Online Label Aggregation: A Variational Bayesian Approach
  Noisy labeled data is more a norm than a rarity for crowd sourced contents.
It is effective to distill noise and infer correct labels through aggregation
results from crowd workers. To ensure the time relevance and overcome slow
responses of workers, online label aggregation is increasingly requested,
calling for solutions that can incrementally infer true label distribution via
subsets of data items. In this paper, we propose a novel online label
aggregation framework, BiLA, which employs variational Bayesian inference
method and designs a novel stochastic optimization scheme for incremental
training. BiLA is flexible to accommodate any generating distribution of labels
by the exact computation of its posterior distribution. We also derive the
convergence bound of the proposed optimizer. We compare BiLA with the state of
the art based on minimax entropy, neural networks and expectation maximization
algorithms, on synthetic and real-world data sets. Our evaluation results on
various online scenarios show that BiLA can effectively infer the true labels,
with an error rate reduction of at least 10 to 1.5 percent points for synthetic
and real-world datasets, respectively.
