Generative power of a protein language model trained on multiple
  sequence alignments
  Computational models starting from large ensembles of evolutionarily related
protein sequences capture a representation of protein families and learn
constraints associated to protein structure and function. They thus open the
possibility for generating novel sequences belonging to protein families.
Protein language models trained on multiple sequence alignments, such as MSA
Transformer, are highly attractive candidates to this end. We propose and test
an iterative method that directly uses the masked language modeling objective
to generate sequences using MSA Transformer. We demonstrate that the resulting
sequences generally score better than those generated by Potts models, and even
than natural sequences, for homology, coevolution and structure-based measures.
Moreover, MSA Transformer better reproduces the higher-order statistics and the
distribution of sequences in sequence space of natural data than Potts models,
although Potts models better reproduce first- and second-order statistics. MSA
Transformer is thus a strong candidate for protein sequence generation and
protein design.
