Protein Folding Neural Networks Are Not Robust
  Deep neural networks such as AlphaFold and RoseTTAFold predict remarkably
accurate structures of proteins compared to other algorithmic approaches. It is
known that biologically small perturbations in the protein sequence do not lead
to drastic changes in the protein structure. In this paper, we demonstrate that
RoseTTAFold does not exhibit such a robustness despite its high accuracy, and
biologically small perturbations for some input sequences result in radically
different predicted protein structures. This raises the challenge of detecting
when these predicted protein structures cannot be trusted. We define the
robustness measure for the predicted structure of a protein sequence to be the
inverse of the root-mean-square distance (RMSD) in the predicted structure and
the structure of its adversarially perturbed sequence. We use adversarial
attack methods to create adversarial protein sequences, and show that the RMSD
in the predicted protein structure ranges from 0.119\r{A} to 34.162\r{A} when
the adversarial perturbations are bounded by 20 units in the BLOSUM62 distance.
This demonstrates very high variance in the robustness measure of the predicted
structures. We show that the magnitude of the correlation (0.917) between our
robustness measure and the RMSD between the predicted structure and the ground
truth is high, that is, the predictions with low robustness measure cannot be
trusted. This is the first paper demonstrating the susceptibility of
RoseTTAFold to adversarial attacks.
