Toward a Perspectivist Turn in Ground Truthing for Predictive Computing
  Most Artificial Intelligence applications are based on supervised machine
learning (ML), which ultimately grounds on manually annotated data. The
annotation process is often performed in terms of a majority vote and this has
been proved to be often problematic, as highlighted by recent studies on the
evaluation of ML models. In this article we describe and advocate for a
different paradigm, which we call data perspectivism, which moves away from
traditional gold standard datasets, towards the adoption of methods that
integrate the opinions and perspectives of the human subjects involved in the
knowledge representation step of ML processes. Drawing on previous works which
inspired our proposal we describe the potential of our proposal for not only
the more subjective tasks (e.g. those related to human language) but also to
tasks commonly understood as objective (e.g. medical decision making), and
present the main advantages of adopting a perspectivist stance in ML, as well
as possible disadvantages, and various ways in which such a stance can be
implemented in practice. Finally, we share a set of recommendations and outline
a research agenda to advance the perspectivist stance in ML.
