Dataset Bias in the Natural Sciences: A Case Study in Chemical Reaction
  Prediction and Synthesis Design
  Datasets in the Natural Sciences are often curated with the goal of aiding
scientific understanding and hence may not always be in a form that facilitates
the application of machine learning. In this paper, we identify three trends
within the fields of chemical reaction prediction and synthesis design that
require a change in direction. First, the manner in which reaction datasets are
split into reactants and reagents encourages testing models in an
unrealistically generous manner. Second, we highlight the prevalence of
mislabelled data, and suggest that the focus should be on outlier removal
rather than data fitting only. Lastly, we discuss the problem of reagent
prediction, in addition to reactant prediction, in order to solve the full
synthesis design problem, highlighting the mismatch between what machine
learning solves and what a lab chemist would need. Our critiques are also
relevant to the burgeoning field of using machine learning to accelerate
progress in experimental Natural Sciences, where datasets are often split in a
biased way, are highly noisy, and contextual variables that are not evident
from the data strongly influence the outcome of experiments.
