Delfos: deep learning model for prediction of solvation free energies in generic organic solvents
Prediction of aqueous solubilities or hydration free energies is an extensively studied area in machine learning applications in chemistry since water is the sole solvent in the living system. However, for nonaqueous solutions, few machine learning studies have been undertaken so far despite the fact that the solvation mechanism plays an important role in various chemical reactions. Here, we introduce Delfos (deep learning model for solvation free energies in generic organic solvents), which is a novel, machinelearning-based QSPR method which predicts solvation free energies for various organic solute and solvent systems. A novelty of Delfos involves two separate solvent and solute encoder networks that can quantify structural features of given compounds via word embedding and recurrent layers, augmented with the attention mechanism which extracts important substructures from outputs of recurrent neural networks. As a result, the predictor network calculates the solvation free energy of a given solventsolute pair using features from encoders. With the results obtained from extensive calculations using 2495 solute-solvent pairs, we demonstrate that Delfos not only has great potential in showing accuracy comparable to that of the state-of-the-art computational chemistry methods, but also offers information about which substructures play a dominant role in the solvation process.
Introduction<!>Word embedding<!>Encoder-predictor network<!>Computational setup and results<!>Transferability of the model for new compounds<!>Visualization of the attention mechanism<!>Conclusions<!>Conflicts of interest
<p>The most common strategies to predict biological or physicochemical properties of chemical compounds are ab initio quantum mechanical approaches [1][2][3][4][5][6][7][8][9] like Hartree-Fock (HF) or density functional theory (DFT) and the molecular dynamics (MD) simulation method based on classical Newtonian and statistical mechanics. [10][11][12][13] These methods with precisely dened theoretical backgrounds have been successfully used in calculating various features of chemical compounds. However, such methods have limitations in computational resources and time costs since they require an enormous amount of numerical calculations. As an alternative, recent successes in the machine learning (ML) technique and its implementation in cheminformatics are promoting broad applications of ML in chemical studies. Quantitative structure-activity relationship (QSAR) or quantitative structure-property relationship (QSPR) analysis is one of such techniques which predict various properties of a given compound from its empirical or structural features. 14,15 The underlying architecture of QSAR/QSPR consists of two elementary mathematical functions. 15 One is the encoding function, which encodes the chemical structure of the given compound into a molecular descriptor. The other, the mapping function, predicts the target property (or activity) that we intend to nd out using the encoded descriptor.</p><p>There have been various molecular descriptors proposed to represent structural features of compounds efficiently. For example, we can feature a given molecule with simple enumerations of empirical properties like molecular weights, rotatable bonds, the number of hydrogen bond donors and acceptors, or some pre-experimental or pre-calculated properties. 16 On the other hand, molecular ngerprints, which are another option, are commonly used in cheminformatics to estimate the chemical 'difference' between more than two compounds. 17 They usually have a xed size of a binary sequence and are easily obtainable from SMILES with pre-dened criteria. Graphical representation of molecules based on graph theory is another major encoding method in QSAR/ QSPR which has received great attention in recent days. 18,19 It has exhibited outstanding prediction performances in diverse chemical or biophysical properties. 20 The mapping function extracts properties which we want to know about from encoded molecular features of the given compound via a classication or regression method. We can use any suitable machine learning method for mapping functions 15,20 such as random forests (RF), support vector machines (SVM), neural networks (NNs), and so on. Among these diverse technical options, NN seems to be the method which has shown the most rapid advances in recent years, 16,[21][22][23][24][25] on the strength of the theoretical advances 26 and evolution of computational power. Many studies have already been performed to show that various chemical or biophysical properties of compounds are obtainable from the QSAR/QSPR combined with machine learning techniques. 16,[20][21][22][23][24][25]27,28 Solvation is one of the most fundamental processes occurring in chemistry, and many theoretical and computational studies have been performed to calculate solubilities or solvation free energies using a variety of methodologies. 29,30 For example, we can roughly guess solubilities using solvation parameters, but solvation parameters only provide the relative order, not the quantitative value. 31 The general solubility equation (GSE) enables us to calculate solubilities from some empirical parameters, but it only provides solubilities for aqueous solutions. 32 Ab initio [1][2][3][4][5][6][7] or MD simulations [10][11][12][13] provide us with more concrete, accurate results and more in-depth knowledge about the solvation mechanism, but they have practical limitations due to high usage of computational resources as mentioned before.</p><p>Recent studies demonstrated that QSPR with ML successfully predicts aqueous solubilities or hydration free energies of diverse solutes. 16,20,21,25,33,34 They also proved that ML guarantees faster calculations than computer simulations and more precise estimations than GSE estimation; a decent number of models showed accuracies comparable to ab initio solvation models. 20 However, the majority of QSPR predictions for solubilities have been limited to cases of aqueous solutions. For non-aqueous solutions, few studies have been undertaken to predict the solubility despite the fact that predicting solubilities plays an important role in the development of varied elds of chemistry, e.g., organic synthesis, 35 electrochemical reactions in batteries, 36 and so on.</p><p>In the present work, we introduce Delfos (deep learning model for solvation free energies in generic organic solvents), which is a QSPR model combined with a recurrent neural network (RNN) model. Delfos is specialized in predicting solvation free energies of organic compounds in various solvents, and the model has three primary sub-neural networks: the solvent and solute encoder networks and the predictor network. For basic featurization of a given molecule, we use the word embedding technique. 34,37 We calculate solvation energies of 2495 pairs of 418 solutes and 91 solvents 38 and demonstrate that our model shows a performance as good as that of the best available quantum chemical methods 2,6,8,11 when the neural network is trained with a sufficient chemical database.</p><p>The rest of the present paper is outlined as follows: Section 2 describes the embedding method for the molecular structure and overall architecture of the neural network. In Section 3, we mainly compare the performance of Delfos with both MD and ab initio simulation strategies 3,6,8,11 and discuss database sensitivity using the cluster cross-validation method. We also visualize important substructures in solvation via the attention mechanism. In the last section, we conclude our work.</p><!><p>Natural language processing (NLP) is one of most cutting-edge subelds of computer science in varied applications of machine learning and neural networks. 37,[39][40][41][42] To process human languages using computers, we need to encode words and sentences and extract their linguistic properties. The process is commonly implemented via the word embedding method. 37,39 To perform this task, unsupervised learning schemes such as skipgram and continuous bag of words (CBOW) algorithms generate a vector representation of the given word in an arbitrary vector space. 37,39 If the corresponding vector space is well-dened, one can deduce the semantic or syntactic features of the given word from the position of the embedded vector, and the inner product of two vectors that correspond to two different words provides information about their linguistic relations.</p><p>It is worthwhile to note that we can employ the embedding technique for chemical or biophysical processes if we consider an atom or a substructure to be a word and a compound to be a sentence. 33,34,43 In this case, positions of molecular substructures in the embedded vector space represent their chemical and physical properties, instead of linguistic information. There are already bio-vector models 43 that have been developed which encode sequences of proteins or DNAs, and atomic-vector embedding models have been introduced recently to encode structural features of chemical compounds. 33,34 Mol2Vec is one of such embedding techniques, and it generates vector representations of a given molecule from the molecular sentence. 34 To make molecular sentences, Mol2Vec uses the Morgan algorithm 44 that classies identical atoms in the molecule. The algorithm is commonly used to generate ECFPs, 45 which are the de facto standard in cheminformatics, 17 and it creates identiers of the given atom from the chemical environment in which the atom is positioned. An atom may have multiple identiers depending on the pre-set maximum value of the radius r max , which denotes the maximum topological distance between the given atom and its neighboring atoms. The atom itself is identied by r ¼ 0, and additional substructure identiers for adjacent atoms are denoted by r ¼ 1 (nearest neighbor), r ¼ 2 (next nearest neighbor), and so on. Since Mol2Vec has demonstrated promising performances in several applications of QSAR/QSPR, 34 Delfos uses Mol2Vec as the primary encoding means. The schematic illustration of the embedding procedure for acetonitrile is shown in Fig. 1.</p><!><p>As shown in Fig. 2, the fundamental architecture of Delfos involves three sub-neural networks: the solvent and the solute encoders extract dominant structural features of the given compound from SMILES strings, while the predictor calculates the solvation energy of the given solvent-solute pair from their encoded features.</p><p>The primary architecture of the encoder is based on two bidirectional recurrent neural networks (BiRNNs). 46 The network is designed for handling sequential data and we consider the molecular sentence [x 1 , ., x N ] to be a sequence of embedded substructures, x i . RNNs may have a failure when input sequences are lengthy; gradients of the loss function can be diluted or amplied because of accumulated precision error from the backpropagation process. 47 The excessive or restrained gradient may cause a decline in learning performance, and we call these two problems vanishing or exploding gradients. To overcome these limits which stem from lengthy input sequences, (copy) one may consider using both the forwarddirectional RNN ðRNN ! Þ and backward-directional RNN ðRNN Þ within a single layer:</p><p>In eqn (1), x i is the embedded atomic vector of a given molecule, h i ! and h i are the hidden state outputs of each recurrent unit, and h i ¼ h i ! ; h i represents the concatenation of two hidden states, respectively. More advanced versions of RNN, like the long short-term memory 48 (LSTM) or gated recurrent unit 49 (GRU) networks, are widely used to handle lengthy input sequences. They introduce gates in each RNN cell state to memorize important information of the previous cell state and minimize vanishing and exploding gradient problems.</p><p>Aer the RNN layers, the molecular sentences of both the solvent X ¼ [x 1 , ., x N ] and the solute Y ¼ [y 1 , ., y M ] are converted to hidden states,</p><p>respectively. Each hidden state is then inputted into the shared attention layer and weighted. The attention mechanism, which was originally proposed to enhance performances of a machine translator, 40 is an essential technique in diverse NLP applications nowadays. 41,42 Principles of the attention start from the denition of the score function of hidden states and its normalization with the somax function:</p><p>There are various score functions that have been introduced to achieve efficient predictions, [40][41][42] and among them we use Luong's dot-product attention 42 in eqn (2c) as a score function since it is computationally efficient. The solvent context, P ¼ aG denotes an emphasized hidden state H with the attention alignment, a. We also obtain the solute context Q using the same procedure. The context weighted from the attention layer is an L Â 2D matrix, where L is the sequence length and D is the dimensions of two RNN hidden layers since we use a bidirectional RNN (BiRNN). Two max-pooling layers, which are the last part of each encoder, reduce contexts H, G, P, and Q to 2Ddimensional feature vectors u and v: 42</p><p>The predictor has a single fully connected perceptron layer with a rectier unit (ReLU) and an output layer. It uses the concatenated feature of the solvent and solute [u;v] as an input. The overall architecture of our model is shown in Fig. 2. We also consider encoders without RNNs and attention layers in order to quantify the impact of these layers on prediction performances of the network; each encoding network contains only the embedding layer and is directly connected to the MLP layer. The solvent and solute features are simple summations of atomic vectors, u ¼  model was initially used for gradient boosting (GBM) regression analysis for aqueous solubilities and toxicities. 34 3 Results and discussion</p><!><p>We use the Minnesota solvation database 38 (MNSOL) as the dataset over which we train and test, and it provides 3037 experimental measures of free energies of solvation and transfer energies for 790 unique solutes in 92 solvents. Because the MNSOL only contains common names of compounds, we perform an automated search process using the PubChemPy 50 script and obtain SMILES strings of compounds from the PubChem database. There are 363 results for charged solutes and 144 results for transfer free energies in the MNSOL which are excluded from the machine learning dataset, and 35 results of solvent-solute combinations are not valid in PubChem. We nally prepare SMILES specications of 2495 solutions for 418 solutes and 91 solvents for the machine learning input.</p><p>For implementation of the neural networks, we use the Keras 2.2.4 framework 51 with TensorFlow 1.12 backend. 52 At the very rst stage, the Morgan algorithm for r ¼ 0 and r ¼ 1 generates molecular sentences of the solvent and solute from their SMILES strings. Then the given molecular sentence is embedded into a sequence of 300-dimensional substructure vectors using the pre-trained Word2Vec model available at https://github.com/samoturk/mol2vec, which contains information on $20 000 000 compounds and $20 000 substructures from ZINC and ChEMBL databases. 34 We consider BiLSTM and BiGRU layers in both solvent and solute encoders to compare their performances. Since our model is a regression problem, we use mean squared error (MSE) as the loss function.</p><p>We employ 10-fold cross-validation (CV) for secure representation of the test data because the dataset we use has a limited number of experimental measures; the total dataset is uniformly and randomly split into 10 subsets, and we iteratively choose one of the subsets as a test set and the training run uses the remaining 9 subsets. Consequentially, a 10-fold CV task performs 10 independent training and test runs, and relative sizes of the training and test sets are 9 to 1. We use Scikit-Learn library 53 to implement the CV task and perform an extensive grid search for tuning hyperparameters: learning algorithms, learning rates, and dimensions of hidden layers. We select the stochastic gradient descent (SGD) algorithm with Nesterov momentum, whose learning rate is 0.0002 and momentum is 0.9. Optimized hidden dimensions are 150 for recurrent layers and 2000 for the fully connected layer. To minimize the variance of the test run, we take averages for all results over 9 independent random CVs, split from different random states.</p><p>Solvation free energies calculated from the MNSOL using attentive BiRNN encoders are exhibited in Fig. 3 and 4. Prediction errors for the BiLSTM model are AE0.57 kcal mol À1 in RMSE and AE0.30 kcal mol À1 in MAE, and the Pearson correlation coefficient R 2 ¼ 0.96, while results from the BiGRU model indicate that there is no meaningful difference between the two recurrent models. The encoder without BiRNN and attention Fig. 3 Benchmark chart for three kinds of encoder networks, for two metrics (MAE and RMSE). The BiLSTM and the BiGRU models show no significant differences, while they make relatively inaccurate predictions without recurrent networks. All results are averaged over 9 independent test runs and black lines on top of boxes denote variances. We cannot directly compare our results with those of other ML models because Delfos is the rst ML-based study using the MNSOL database. Nonetheless, several studies on aqueous systems have previously calculated solubilities or hydration free energies using various ML techniques and molecular descriptors. 16,20,21,25,33,34 For comparison, we have tested our neural network model for the hydration free energy. A benchmark study from Wu et al. 20 provides hydration energies of 642 small molecules in a group of QSPR/ML models. Their RMSEs were up to 1.15 kcal mol À1 while our prediction from the BiLSTM encoder attains 1.19 kcal mol À1 for the same dataset and split method (see the ESI †). This result suggests that our neural network model guarantees considerably good performances even in a specic solvent of water.</p><p>Meanwhile, for studies which are not ML-based, there are several results from both classical and quantum-mechanical simulation studies that use the MNSOL as the reference database. [3][4][5][6]8,11,13 In Table 1, we choose two DFT studies which employ several widely used QM solvation models 3,8 for comparison with our proposed ML model: solvation model 8/ 12 (SM8/SM12), the solvation model based on density (SMD), and the full/direct conductor-like screening model for realistic solvation (COSMO-RS/D-COSMO-RS). While all of these QM methods exhibited excellent performances when considering a chemical accuracy of 1.0 kcal mol À1 , full COSMO-RS is a noteworthy solvation model since it is believed to be a state-of-the-art method which shows the best accuracy. 9 This is realized by statistical thermodynamics treatment on the polarization charge densities, which helps COSMO-RS with making successful predictions even in polar solvents where the key idea of the dielectric continuum solvation collapses. 1,7,9 Resultingly, COSMO-RS calculations with the BP86 functional and TZVP basis set achieved 0.52 kcal mol À1 for 274 aqueous solvents, 0.41 kcal mol À1 for 2072 organic solvents, and 0.43 kcal mol À1 for the full dataset in mean absolute error. 8 For the proposed ML models, Delfos with BiLSTM shows a comparable accuracy in the water solvent, for which MAE is 0.64 kcal mol À1 . Delfos makes much better predictions in nonaqueous organic solvents; machine learning for 2121 nonaqueous systems results in 0.24 kcal mol À1 , which is 44% that of SM12CM5 and 59% that of COSMO-RS. However, one may argue that K-fold CV from random split does not produce the real prediction accuracy of the model. That is, the random-CV results only indicate the accuracy for trained or practiced chemical structures. Accordingly, one may ask the following questions. For example, will the ML model ensure comparable prediction accuracy in "structurally" new compounds? What happens if the ML model cannot learn sufficiently varied chemical structures? We will discuss these questions in the next section.</p><!><p>Since our study uses techniques of machine learning with empirical data from experimental measures, there is a likelihood that Delfos would not guarantee prediction accuracy for structurally new solvents or solutes which are not present in the dataset, although the MNSOL contains a considerable number of commonly used solvents and solutes. 38 In order to investigate this potential issue, we perform another training and test run with cluster cross-validation, 54,55 instead of using the randomsplit CV. As a start, we individually obtain 10 clusters for solvents and solutes using the K-mean clustering algorithm and the molecular vector. The molecular vector is a simple summation of substructure vectors used for the simple MLP model without RNN encoders: 34 u ¼ X N i x i for solvents and v ¼ X M i y i for solutes, respectively. Then, we iteratively perform Table 1 Comparison between encoder-predictor networks and various quantum-mechanical solvation models for aqueous and non-aqueous solutions. The error metric is MAE and kcal mol À1 . Data in bold are our results, while QM results are taken from the work of Marenich et al. 3 55 reported that the split method based on the clustering exhibits an apparent degradation of prediction performances in various properties; we nd that our proposed model exhibits a similar tendency as well. For the BiLSTM encoder model, increments of MAE are 0.52 kcal mol À1 for the solvent clustering and 0.69 kcal mol À1 for the solute clustering. The reason why the random K-fold CV exhibits superior performances is obvious; if we have a pair ðA; BÞ of solvents A and solutes B in the test set and the training set has ðA; CÞ and ðD; BÞ pairs, then both ðA; CÞ and ðD; BÞ could enhance the prediction accuracy of ðA; BÞ. However, the clustering limits the location of a specic compound, and pairs of specic solvents or solutes should be either in the test set or the training set.</p><p>For an additional comparison, Table 2 also contains results taken from SMD with semi-empirical methods, 6 pure COSMO, COSMO-RS, 8 and classical molecular dynamics 11 for four organic solvents: toluene (C 6 H 5 CH 3 ), chloroform (CHCl 3 ), acetonitrile (CH 3 CN), and dimethyl sulfoxide ((CH 3 ) 2 SO), respectively. Although the MD is based on classical dynamics, the results of the generalized amber force eld (GAFF) tell us that an explicit solvation model with a suitable force eld could make considerably good predictions. The bottom line of cluster CV is if the dataset for training contains at least one side of the solvent-solvent pair we want to estimate its solvation free energy, the expectation error of Delfos lies within a chemical accuracy of 1.0 kcal mol À1 , which is the general error of the computer simulation scheme. Also, results for four organic solvents demonstrate that predictions from the cluster CV have an accuracy that is comparable with that of MD simulations using an AMOEBA polarizable force eld. 11 Results from the cluster CV highlight the necessity for discussion on the importance of database preparation. As described earlier, the cluster CV causes a considerable increase Table 2 Prediction accuracy of the random-split CV, the solvent and solute cluster CV using the K-mean algorithm, and several theoretical solvation models for four different organic solvents: toluene (C 6 H 5 CH 3 ), chloroform (CHCl 3 ), acetonitrile (CH 3 CN), and dimethyl sulfoxide ((CH 3 ) 2 SO), respectively. Units of MAE and RMSE are kcal mol À1 in prediction error, and we suspect that the degradation mainly comes from the decline in the diversity of the training set.</p><p>Namely, the number of substructures that the neural network learns in the training process is not as many as the random CV if we use the cluster CV. To prove this speculation, we dene unique substructures, which are substructures that only exists in the test cluster. As shown in Fig. 5, in the solute cluster CV, the MAE for 1226 pairs which don't have any unique substructures in solutes is 0.54 kcal mol À1 , while the prediction error for the remaining 1269 solutions is 1.64 kcal mol À1 . The solvent cluster CV shows more extreme results: the MAE for 374 aqueous solvents is 2.48 kcal mol À1 , while non-aqueous solvents exhibit 0.52 kcal mol À1 in contrast. We believe that the outlying behavior of water is due to distinctive nature. Water has only one unique substructure since oxygen atom does not have any neighbors. So the solvent clustering makes the network unable to learn the structure of water in indirect ways, resulting in prediction failure. This logic tells us that the most critical thing is securing of the training dataset which contains as many kinds of solvents and solutes as possible. We believe that computational approaches would be as helpful as experimental measures for enriching structural diversity of the training data, given recent advances on QM solvation models 2,3,8 such as COSMO-RS. Furthermore, since there are 418 solutes and 91 solvents in the dataset used, 38 which make up 38 038 possible pairs, we expect Delfos and MNSOL to guarantee similar precision levels with the random CV for numerous systems.</p><!><p>A useful aspect of the attention mechanism is that the model provides not only the prediction value of solvation energy of a given input but also a clue to why the neural network makes such a prediction based on the correlations between recurrent hidden states. 25,33,41 In this section, we visualize how the attention layer operates, and verify how such correlations correspond well to chemical intuitions for inter-molecular interactions. The matrix of attention alignments, a, from eqn (2a) indicates which substructures in the given solvent and solute are strongly correlated with each other so that they play dominant roles in determining their solvation energy. In Fig. ). The scheme for visualizing attention alignments is as follows: (i) rst, we calculate the average alignment hai j of each substructure j of the solute over the entire solvent structure {i},</p><p>(ii) Then, we get relative amounts of averaged alignments [a1, ., aM] by dividing by the maximum value, aj ¼ hai j /max(hai 1 , ., hai M ). (iii) Also, since the embedding algorithm used generates two substructure vectors per atom, we individually visualize two alignment maps, [a1, a3, ., aM À1 ] (for r ¼ 0) and [a2, a4, ., aM] (for r ¼ 1) for more simple and intuitive illustration. (iv) Finally, the color representation of each atom in Fig. 6 denotes the amount of aj; the neural network judges that red-colored substructures (higher aj) in the solute are more "similar" to the solvent and the model puts more weight on them during the prediction task. In contrast, green-colored substructures have a lower aj, which means they do not share similarities with the solvent molecule as much as the red-colored ones.</p><p>Overall the results in Fig. 6 imply that the chemical similarity taken from the attention layer has a signicant connection to a fundamental knowledge of chemistry like polarity or hydrophilicity. Each alcoholic solvent has one hydrophilic -OH group, and this results in increasing contributions of the nitro group in the solute as hydrocarbon chains of alcohols shorten. For the acetonitrile-nitromethane solution, the attention mechanism reects the highest contributions of -NO 2 groups due to the strong polarity and aprotic nature of the solvent. Although the attention mechanism seems to reproduce molecular interactions in a faithful way, we nd that there is a defective prediction which does not agree with chemical knowledge. Two oxygen atoms ]O and -O À in the nitro group are indistinguishable due to the resonance structure; thus they must have equivalent contributions in any solvent, but we nd that they show different attention scores in our model. We believe that these problems occur because the SMILES string of nitromethane (C[N+](]O)[O-]) does not encode the resonance effect Fig. 5 Results of cross-validation tasks using the K-mean clustering algorithm for (a) solutes and (b) solvents. We conclude that unique substructures in the given compounds are the main cause for the decline in prediction accuracy. Each encoder network includes a BiLSTM layer and we use the same hyperparameters which are optimized in the random CV task.</p><p>in the nitro group. Indeed, the Morgan algorithm generates different identiers for two oxygen atoms in the nitro group, [864 942 730, 2 378 779 377] for ]O and [864 942 795, 2 378 775 366] for -O À . The absence of resonance might be a problem worth considering when one intends to use word embedding models with SMILES strings, 33,34,55 although estimated solvation energies for nitromethane from the BiLSTM model are within a moderate error range as shown in Fig. 6.</p><!><p>In the present study, we introduced a QSPR regression neural network solvation energy estimation that is inspired by NLP. The proposed model has two separate encoder neural networks for solvents and solutes and a predictor neural network. Each encoder neural network is designed to encode the chemical structure of an input compound into the feature vector of a specic size. The encoding procedure is accomplished using the Mol2Vec embedding model 34 and recurrent neural networks with the attention mechanism. [40][41][42] The predictor neural network with fully connected MLP calculates the solvation free energy of a given solvent-solute pair using the feature vectors from encoders.</p><p>We performed extensive calculations on 2495 experimental values of solvation energies taken from the MNSOL database. 38 From the random-CV task, we obtained mean averaged errors in solvation free energy of Delfos using BiLSTM as 0.64 kcal mol À1 for aqueous systems and 0.24 kcal mol À1 for non-aqueous systems. Our results demonstrate that the proposed model exhibits excellent prediction accuracy which is comparable with that of several well-known QM solvation models 3,8 when the neural network is trained with sufficiently varied chemical structures, while the MLP model which does not contain recurrent or attention layers showed relatively decient performances. A decline in performances of about 0.5 to 0.7 kcal mol À1 at the cluster CV tasks represents the accuracy for a structurally new compound, suggesting the importance of preparation of ML databases even though Delfos still demonstrates comparable predictions with some theoretical approaches such as MD using the AMOEBA force eld 11 or DFT with pure COSMO. 8 The score matrix taken from the attention mechanism gives us an interaction map between the atoms and substructure; our model not only provides a simple estimation of target property but also offers important pieces of information about which substructures play a dominant role in solvation processes.</p><p>One of the most useful advantages of ML is exibility; a single model can be used to learn and predict various databases. 20 Also, our model may be applied to predict various chemical, physical, or biological properties especially focused on interactions between more than two different chemical species. One of the possible applications that we can consider is the prediction of chemical affinity and the possibility of various chemical reactions. 56 Room-temperature ionic liquids might be another potential research topic became the interplay between cations and anions dominates their various properties, e.g., toxicity 57 or electrochemical properties in supercapacitors. 58,59 Thus, we expect that Delfos will be helpful in many further studies, and not only localized to the prediction of solvation energies.</p><!><p>There are no conicts to declare.</p>