Infusing Linguistic Knowledge of SMILES into Chemical Language Models
  The simplified molecular-input line-entry system (SMILES) is the most popular
representation of chemical compounds. Therefore, many SMILES-based molecular
property prediction models have been developed. In particular,
transformer-based models show promising performance because the model utilizes
a massive chemical dataset for self-supervised learning. However, there is no
transformer-based model to overcome the inherent limitations of SMILES, which
result from the generation process of SMILES. In this study, we grammatically
parsed SMILES to obtain connectivity between substructures and their type,
which is called the grammatical knowledge of SMILES. First, we pretrained the
transformers with substructural tokens, which were parsed from SMILES. Then, we
used the training strategy 'same compound model' to better understand SMILES
grammar. In addition, we injected knowledge of connectivity and type into the
transformer with knowledge adapters. As a result, our representation model
outperformed previous compound representations for the prediction of molecular
properties. Finally, we analyzed the attention of the transformer model and
adapters, demonstrating that the proposed model understands the grammar of
SMILES.
