Meta-Learning Approaches for a One-Shot Collective-Decision Aggregation:
  Correctly Choosing how to Choose Correctly
  Aggregating successfully the choices regarding a given decision problem made
by the multiple collective members into a single solution is essential for
exploiting the collective's intelligence and for effective crowdsourcing. There
are various aggregation techniques, some of which come down to a simple and
sometimes effective deterministic aggregation rule. However, it has been shown
that the efficiency of those techniques is unstable under varying conditions
and within different domains. Other methods mainly rely on learning from the
decision-makers previous responses or the availability of additional
information about them. In this study, we present two one-shot
machine-learning-based aggregation approaches. The first predicts, given
multiple features about the collective's choices, including meta-cognitive
ones, which aggregation method will be best for a given case. The second
directly predicts which decision is optimal, given, among other things, the
selection made by each method. We offer a meta-cognitive feature-engineering
approach for characterizing a collective decision-making case in a
context-sensitive fashion. In addition, we offer a new aggregation method, the
Devil's-Advocate aggregator, to deal with cases in which standard aggregation
methods are predicted to fail. Experimental results show that using either of
our proposed approaches increases the percentage of successfully aggregated
cases (i.e., cases in which the correct answer is returned) significantly,
compared to the uniform application of each rule-based aggregation method. We
also demonstrate the importance of the Devil's Advocate aggregator.
