Making Graph Neural Networks Worth It for Low-Data Molecular Machine
  Learning
  Graph neural networks have become very popular for machine learning on
molecules due to the expressive power of their learnt representations. However,
molecular machine learning is a classically low-data regime and it isn't clear
that graph neural networks can avoid overfitting in low-resource settings. In
contrast, fingerprint methods are the traditional standard for low-data
environments due to their reduced number of parameters and manually engineered
features. In this work, we investigate whether graph neural networks are
competitive in small data settings compared to the parametrically 'cheaper'
alternative of fingerprint methods. When we find that they are not, we explore
pretraining and the meta-learning method MAML (and variants FO-MAML and ANIL)
for improving graph neural network performance by transfer learning from
related tasks. We find that MAML and FO-MAML do enable the graph neural network
to outperform models based on fingerprints, providing a path to using graph
neural networks even in settings with severely restricted data availability. In
contrast to previous work, we find ANIL performs worse that other meta-learning
approaches in this molecule setting. Our results suggest two reasons: molecular
machine learning tasks may require significant task-specific adaptation, and
distribution shifts in test tasks relative to train tasks may contribute to
worse ANIL performance.
