Benchmarking Active Learning Strategies for Materials Optimization and
  Discovery
  Autonomous physical science is revolutionizing materials science. In these
systems, machine learning controls experiment design, execution, and analysis
in a closed loop. Active learning, the machine learning field of optimal
experiment design, selects each subsequent experiment to maximize knowledge
toward the user goal. Autonomous system performance can be further improved
with implementation of scientific machine learning, also known as inductive
bias-engineered artificial intelligence, which folds prior knowledge of
physical laws (e.g., Gibbs phase rule) into the algorithm. As the number,
diversity, and uses for active learning strategies grow, there is an associated
growing necessity for real-world reference datasets to benchmark strategies. We
present a reference dataset and demonstrate its use to benchmark active
learning strategies in the form of various acquisition functions. Active
learning strategies are used to rapidly identify materials with optimal
physical properties within a ternary materials system. The data is from an
actual Fe-Co-Ni thin-film library and includes previously acquired experimental
data for materials compositions, X-ray diffraction patterns, and two functional
properties of magnetic coercivity and the Kerr rotation. Popular active
learning methods along with a recent scientific active learning method are
benchmarked for their materials optimization performance. We discuss the
relationship between algorithm performance, materials search space complexity,
and the incorporation of prior knowledge.
