GeneDisco: A Benchmark for Experimental Design in Drug Discovery
  In vitro cellular experimentation with genetic interventions, using for
example CRISPR technologies, is an essential step in early-stage drug discovery
and target validation that serves to assess initial hypotheses about causal
associations between biological mechanisms and disease pathologies. With
billions of potential hypotheses to test, the experimental design space for in
vitro genetic experiments is extremely vast, and the available experimental
capacity - even at the largest research institutions in the world - pales in
relation to the size of this biological hypothesis space. Machine learning
methods, such as active and reinforcement learning, could aid in optimally
exploring the vast biological space by integrating prior knowledge from various
information sources as well as extrapolating to yet unexplored areas of the
experimental design space based on available data. However, there exist no
standardised benchmarks and data sets for this challenging task and little
research has been conducted in this area to date. Here, we introduce GeneDisco,
a benchmark suite for evaluating active learning algorithms for experimental
design in drug discovery. GeneDisco contains a curated set of multiple publicly
available experimental data sets as well as open-source implementations of
state-of-the-art active learning policies for experimental design and
exploration.
