Analysis of Minimax Error Rate for Crowdsourcing and Its Application to
  Worker Clustering Model
  While crowdsourcing has become an important means to label data, there is
great interest in estimating the ground truth from unreliable labels produced
by crowdworkers. The Dawid and Skene (DS) model is one of the most well-known
models in the study of crowdsourcing. Despite its practical popularity,
theoretical error analysis for the DS model has been conducted only under
restrictive assumptions on class priors, confusion matrices, or the number of
labels each worker provides. In this paper, we derive a minimax error rate
under more practical setting for a broader class of crowdsourcing models
including the DS model as a special case. We further propose the worker
clustering model, which is more practical than the DS model under real
crowdsourcing settings. The wide applicability of our theoretical analysis
allows us to immediately investigate the behavior of this proposed model, which
can not be analyzed by existing studies. Experimental results showed that there
is a strong similarity between the lower bound of the minimax error rate
derived by our theoretical analysis and the empirical error of the estimated
value.
