The Scientific Method in the Science of Machine Learning
  In the quest to align deep learning with the sciences to address calls for
rigor, safety, and interpretability in machine learning systems, this
contribution identifies key missing pieces: the stages of hypothesis
formulation and testing, as well as statistical and systematic uncertainty
estimation -- core tenets of the scientific method. This position paper
discusses the ways in which contemporary science is conducted in other domains
and identifies potentially useful practices. We present a case study from
physics and describe how this field has promoted rigor through specific
methodological practices, and provide recommendations on how machine learning
researchers can adopt these practices into the research ecosystem. We argue
that both domain-driven experiments and application-agnostic questions of the
inner workings of fundamental building blocks of machine learning models ought
to be examined with the tools of the scientific method, to ensure we not only
understand effect, but also begin to understand cause, which is the raison
d'\^{e}tre of science.
