An Experience Report on Machine Learning Reproducibility: Guidance for
  Practitioners and TensorFlow Model Garden Contributors
  Machine learning techniques are becoming a fundamental tool for scientific
and engineering progress. These techniques are applied in contexts as diverse
as astronomy and spam filtering. However, correctly applying these techniques
requires careful engineering. Much attention has been paid to the technical
potential; relatively little attention has been paid to the software
engineering process required to bring research-based machine learning
techniques into practical utility. Technology companies have supported the
engineering community through machine learning frameworks such as TensorFLow
and PyTorch, but the details of how to engineer complex machine learning models
in these frameworks have remained hidden.
  To promote best practices within the engineering community, academic
institutions and Google have partnered to launch a Special Interest Group on
Machine Learning Models (SIGMODELS) whose goal is to develop exemplary
implementations of prominent machine learning models in community locations
such as the TensorFlow Model Garden (TFMG). The purpose of this report is to
define a process for reproducing a state-of-the-art machine learning model at a
level of quality suitable for inclusion in the TFMG. We define the engineering
process and elaborate on each step, from paper analysis to model release. We
report on our experiences implementing the YOLO model family with a team of 26
student researchers, share the tools we developed, and describe the lessons we
learned along the way.
