Off-the-shelf deep learning is not enough: parsimony, Bayes and
  causality
  Deep neural networks ("deep learning") have emerged as a technology of choice
to tackle problems in natural language processing, computer vision, speech
recognition and gameplay, and in just a few years has led to superhuman level
performance and ushered in a new wave of "AI." Buoyed by these successes,
researchers in the physical sciences have made steady progress in incorporating
deep learning into their respective domains. However, such adoption brings
substantial challenges that need to be recognized and confronted. Here, we
discuss both opportunities and roadblocks to implementation of deep learning
within materials science, focusing on the relationship between correlative
nature of machine learning and causal hypothesis driven nature of physical
sciences. We argue that deep learning and AI are now well positioned to
revolutionize fields where causal links are known, as is the case for
applications in theory. When confounding factors are frozen or change only
weakly, this leaves open the pathway for effective deep learning solutions in
experimental domains. Similarly, these methods offer a pathway towards
understanding the physics of real-world systems, either via deriving reduced
representations, deducing algorithmic complexity, or recovering generative
physical models. However, extending deep learning and "AI" for models with
unclear causal relationship can produce misleading and potentially incorrect
results. Here, we argue the broad adoption of Bayesian methods incorporating
prior knowledge, development of DL solutions with incorporated physical
constraints, and ultimately adoption of causal models, offers a path forward
for fundamental and applied research. Most notably, while these advances can
change the way science is carried out in ways we cannot imagine, machine
learning is not going to substitute science any time soon.
