Data and its (dis)contents: A survey of dataset development and use in
  machine learning research
  Datasets have played a foundational role in the advancement of machine
learning research. They form the basis for the models we design and deploy, as
well as our primary medium for benchmarking and evaluation. Furthermore, the
ways in which we collect, construct and share these datasets inform the kinds
of problems the field pursues and the methods explored in algorithm
development. However, recent work from a breadth of perspectives has revealed
the limitations of predominant practices in dataset collection and use. In this
paper, we survey the many concerns raised about the way we collect and use data
in machine learning and advocate that a more cautious and thorough
understanding of data is necessary to address several of the practical and
ethical issues of the field.
