Data Representativity for Machine Learning and AI Systems
  Data representativity is crucial when drawing inference from data through
machine learning models. Scholars have increased focus on unraveling the bias
and fairness in the models, also in relation to inherent biases in the input
data. However, limited work exists on the representativity of samples
(datasets) for appropriate inference in AI systems. This paper analyzes data
representativity in scientific literature related to AI and sampling, and gives
a brief overview of statistical sampling methodology from disciplines like
sampling of physical materials, experimental design, survey analysis, and
observational studies. Different notions of a 'representative sample' exist in
past and present literature. In particular, the contrast between the notion of
a representative sample in the sense of coverage of the input space, versus a
representative sample as a miniature of the target population is of relevance
when building AI systems. Using empirical demonstrations on US Census data, we
demonstrate that the first is useful for providing equality and demographic
parity, and is more robust to distribution shifts, whereas the latter notion is
useful in situations where the purpose is to make historical inference or draw
inference about the underlying population in general, or make better
predictions for the majority in the underlying population. We propose a
framework of questions for creating and documenting data, with data
representativity in mind, as an addition to existing datasheets for datasets.
Finally, we will also like to call for caution of implicit, in addition to
explicit, use of a notion of data representativeness without specific
clarification.
