Graph Mining Meets Crowdsourcing: Extracting Experts for Answer
  Aggregation
  Aggregating responses from crowd workers is a fundamental task in the process
of crowdsourcing. In cases where a few experts are overwhelmed by a large
number of non-experts, most answer aggregation algorithms such as the majority
voting fail to identify the correct answers. Therefore, it is crucial to
extract reliable experts from the crowd workers. In this study, we introduce
the notion of "expert core", which is a set of workers that is very unlikely to
contain a non-expert. We design a graph-mining-based efficient algorithm that
exactly computes the expert core. To answer the aggregation task, we propose
two types of algorithms. The first one incorporates the expert core into
existing answer aggregation algorithms such as the majority voting, whereas the
second one utilizes information provided by the expert core extraction
algorithm pertaining to the reliability of workers. We then give a theoretical
justification for the first type of algorithm. Computational experiments using
synthetic and real-world datasets demonstrate that our proposed answer
aggregation algorithms outperform state-of-the-art algorithms.
