Model Positionality and Computational Reflexivity: Promoting Reflexivity
  in Data Science
  Data science and machine learning provide indispensable techniques for
understanding phenomena at scale, but the discretionary choices made when doing
this work are often not recognized. Drawing from qualitative research
practices, we describe how the concepts of positionality and reflexivity can be
adapted to provide a framework for understanding, discussing, and disclosing
the discretionary choices and subjectivity inherent to data science work. We
first introduce the concepts of model positionality and computational
reflexivity that can help data scientists to reflect on and communicate the
social and cultural context of a model's development and use, the data
annotators and their annotations, and the data scientists themselves. We then
describe the unique challenges of adapting these concepts for data science work
and offer annotator fingerprinting and position mining as promising solutions.
Finally, we demonstrate these techniques in a case study of the development of
classifiers for toxic commenting in online communities.
