Transformer-CNN: Fast and Reliable tool for QSAR
  We present SMILES-embeddings derived from the internal encoder state of a
Transformer [1] model trained to canonize SMILES as a Seq2Seq problem. Using a
CharNN [2] architecture upon the embeddings results in higher quality
interpretable QSAR/QSPR models on diverse benchmark datasets including
regression and classification tasks. The proposed Transformer-CNN method uses
SMILES augmentation for training and inference, and thus the prognosis is based
on an internal consensus. That both the augmentation and transfer learning are
based on embeddings allows the method to provide good results for small
datasets. We discuss the reasons for such effectiveness and draft future
directions for the development of the method. The source code and the
embeddings needed to train a QSAR model are available on
https://github.com/bigchem/transformer-cnn. The repository also has a
standalone program for QSAR prognosis which calculates individual atoms
contributions, thus interpreting the model's result. OCHEM [3] environment
(https://ochem.eu) hosts the on-line implementation of the method proposed.
