Fast differentiable DNA and protein sequence optimization for molecular
  design
  Designing DNA and protein sequences with improved function has the potential
to greatly accelerate synthetic biology. Machine learning models that
accurately predict biological fitness from sequence are becoming a powerful
tool for molecular design. Activation maximization offers a simple design
strategy for differentiable models: one-hot coded sequences are first
approximated by a continuous representation which is then iteratively optimized
with respect to the predictor oracle by gradient ascent. While elegant, this
method suffers from vanishing gradients and may cause predictor pathologies
leading to poor convergence. Here, we build on a previously proposed
straight-through approximation method to optimize through discrete sequence
samples. By normalizing nucleotide logits across positions and introducing an
adaptive entropy variable, we remove bottlenecks arising from overly large or
skewed sampling parameters. The resulting algorithm, which we call Fast
SeqProp, achieves up to 100-fold faster convergence compared to previous
versions of activation maximization and finds improved fitness optima for many
applications. We demonstrate Fast SeqProp by designing DNA and protein
sequences for six deep learning predictors, including a protein structure
predictor.
