How To Grade a Test Without Knowing the Answers --- A Bayesian Graphical
  Model for Adaptive Crowdsourcing and Aptitude Testing
  We propose a new probabilistic graphical model that jointly models the
difficulties of questions, the abilities of participants and the correct
answers to questions in aptitude testing and crowdsourcing settings. We devise
an active learning/adaptive testing scheme based on a greedy minimization of
expected model entropy, which allows a more efficient resource allocation by
dynamically choosing the next question to be asked based on the previous
responses. We present experimental results that confirm the ability of our
model to infer the required parameters and demonstrate that the adaptive
testing scheme requires fewer questions to obtain the same accuracy as a static
test scenario.
