Designing Evaluations of Machine Learning Models for Subjective
  Inference: The Case of Sentence Toxicity
  Machine Learning (ML) is increasingly applied in real-life scenarios, raising
concerns about bias in automatic decision making. We focus on bias as a notion
of opinion exclusion, that stems from the direct application of traditional ML
pipelines to infer subjective properties. We argue that such ML systems should
be evaluated with subjectivity and bias in mind. Considering the lack of
evaluation standards yet to create evaluation benchmarks, we propose an initial
list of specifications to define prior to creating evaluation datasets, in
order to later accurately evaluate the biases. With the example of a sentence
toxicity inference system, we illustrate how the specifications support the
analysis of biases related to subjectivity. We highlight difficulties in
instantiating these specifications and list future work for the crowdsourcing
community to help the creation of appropriate evaluation datasets.
