Lessons from Archives: Strategies for Collecting Sociocultural Data in
  Machine Learning
  A growing body of work shows that many problems in fairness, accountability,
transparency, and ethics in machine learning systems are rooted in decisions
surrounding the data collection and annotation process. In spite of its
fundamental nature however, data collection remains an overlooked part of the
machine learning (ML) pipeline. In this paper, we argue that a new
specialization should be formed within ML that is focused on methodologies for
data collection and annotation: efforts that require institutional frameworks
and procedures. Specifically for sociocultural data, parallels can be drawn
from archives and libraries. Archives are the longest standing communal effort
to gather human information and archive scholars have already developed the
language and procedures to address and discuss many challenges pertaining to
data collection such as consent, power, inclusivity, transparency, and ethics &
privacy. We discuss these five key approaches in document collection practices
in archives that can inform data collection in sociocultural ML. By showing
data collection practices from another field, we encourage ML research to be
more cognizant and systematic in data collection and draw from
interdisciplinary expertise.
