Dynamic Task Allocation for Crowdsourcing Settings
  We consider the problem of optimal budget allocation for crowdsourcing
problems, allocating users to tasks to maximize our final confidence in the
crowdsourced answers. Such an optimized worker assignment method allows us to
boost the efficacy of any popular crowdsourcing estimation algorithm. We
consider a mutual information interpretation of the crowdsourcing problem,
which leads to a stochastic subset selection problem with a submodular
objective function. We present experimental simulation results which
demonstrate the effectiveness of our dynamic task allocation method for
achieving higher accuracy, possibly requiring fewer labels, as well as
improving upon a previous method which is sensitive to the proportion of users
to questions.
