{'fraction_non_alphanumeric': 0.02248426480624858, 'fraction_numerical': 0.009327367862288617, 'mean_word_length': 5.326457183976973, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
DEL-Dock: Molecular Docking-Enabled Modeling of DNA-Encoded Libraries

DNA-Encoded Library (DEL) technology has enabled significant advances in hit identification by enabling efficient testing of combinatorially-generated molecular libraries. DEL screens measure protein binding affinity though sequencing reads of molecules tagged with unique DNA-barcodes that survive a series of selection experiments. Computational models have been deployed to learn the latent binding affinities that are correlated to the sequenced count data; however, this correlation is often obfuscated by various sources of noise introduced in its complicated data-generation process. In order to denoise DEL count data and screen for molecules with good binding affinity, computational models require the correct assumptions in their modeling structure to capture the correct signals underlying the data. Recent advances in DEL models have focused on probabilistic formulations of count data, but existing approaches have thus far been limited to only utilizing 2-D molecule-level representations. We introduce a new paradigm, DEL-Dock, that combines ligand-based descriptors with 3-D spatial information from docked protein-ligand complexes. 3-D spatial information allows our model to learn over the actual binding modality rather than using only structured-based information of the ligand. We show that our model is capable of effectively denoising DEL count data to predict molecule enrichment scores that are better correlated with experimental binding affinity measurements compared to prior works. Moreover, by learning over a collection of docked poses we demonstrate that our model, trained only on DEL data, implicitly learns to perform good docking pose selection without requiring external supervision from expensive-to-source protein crystal structures.


Introduction
One key component of early drug discovery is hit identification, or finding molecules with desired activity levels and properties of interest. 1 This task has been commonly approached via high-throughput screening (HTS) techniques, which involve testing a library of molecules against a biological target of interest. While traditional screening techniques do not scale well to large chemical spaces, DNA-encoded library (DEL) technologies provide one avenue towards the desired scale. For instance, while traditional HTS libraries typically contain only ∼50k-5M compounds, DELs enable screening of combinatorially large molecular spaces that allow for testing ∼1M-5B compounds in a single tube. 2,3 By capturing a broad chemical diversity landscape, DEL technology has opened new opportunities in hit identification. [4][5][6][7] DELs are constructed by sequentially assembling molecular building blocks, aka synthons, into molecules tagged with unique DNA-barcode identifiers (Fig. 1). Once synthesized, the library is tested for affinity against a protein of interest through a series of selection experiments. This process, called panning, typically involves spiking the DEL into a solution of the immobilized protein, and washing the resulting mixture for multiple rounds. This procedure leaves members of the library that remain bound to either the target or matrix, which are subsequently identified using next-generation DNA sequencing. The resulting data after bioinformatics processing consists of sparse reads of the DNA and the corresponding molecules; the relative abundance of the identified molecules is, in theory, a reasonable proxy for their binding affinities. However, DEL data from panning experiments contain various sources of noise such as matrix binding, truncation products, unequal initial loads, replicate and sequencing biases . [8][9][10][11] Modeling the data without considering these significant sources of noise can overfit to spurious correlations. Consequently, while DELs have proven to be powerful tools in drug discovery, careful consideration of the appropriate techniques to denoise DEL data is required to discover reliable signals of the underlying small molecule binding affinities.
Several approaches have been developed to tackle DEL modeling from a computational perspective: these existing methods rely on computing an enrichment score for each molecule that is representative of its binding affinity to the protein target. One approach for calculating enrichment scores involves fitting a Poisson distribution to observed on-target and control count data for each molecule and then formulating enrichment scores as ratios of these Poisson parameters. 12 A deficiency in this enrichment metric is that it neglects to include any structural information of the actual DEL molecules. Since molecular structure represents an important aspect for determining protein-ligand interactions, 13,14 other methods incorporate different types of molecular representations into their models. McCloskey et al. 15 use a Graph Neural Network (GNN) to encode molecules, formulating the learning problem as multi-class binary classification over the counts. Recent methods consider hybrid approaches that combine molecular representation learning with probablistic modeling. 8,9,16 Central to these approaches involves modelling observed sequencing counts as originating from latent Poisson or Gamma-Poisson distributions, which are well suited to describe independently distributed count events. This allows the models to incorporate the inherent noise in the data-generating process within the model structure. However, these methods focus on learning only representations of the molecules themselves and omit the rich representation of the 3-D protein-ligand interactions which are critical aspects for describing protein binding.
In order to leverage 3-D spatial data, we turn to molecular docking, which has a long history of being an important tool in structure-based drug discovery. 17 A key component of these docking methods is to define a scoring function that can characterize the binding interactions between a protein and ligand. To generate candidate poses, these methods use the aforementioned scoring function to sample ligand conformations with the goal of ultimately inferring the most likely binding mode. While there have been many advances to docking models and software, in practice, there are still many pitfalls of these methods. 18 For instance, the difficulty of the docking problem can be observed through the low empirical correlation between high scoring docked poses and actual binding affinity. 19 Improvements can be realized through carefully calibrated scoring functions, but tuning these scoring functions is frequently problem-specific and involves substantial domain knowledge. 20-22 ML-enabled scoring functions trained on databases of crystal structures have recently demonstrated improved performance in some settings, 23 however, these approaches are ultimately limited by scarce and expensive-to-generate crystal structures. Even though docking generates noisy ligand conformations, these docking techniques can provide a useful distribution of likely binding poses which we will exploit in our new ideas for modeling DEL count data.
Towards the goal of more holistic DEL models, we propose DEL-Dock, a model that directly learns a joint protein-ligand representation by synthesizing multi-modal information within a unified probabilistic framework to learn enrichment scores. Our approach combines molecule-level descriptors with spatial information from docked protein-ligand complexes to explain sequencing counts. We delineate contributions from spurious matrix binding and target protein binding to better separate the signal from noise in the data. Additionally, our model can learn to rank a collection of poses without explicit supervision of pose scores by only training on the count data. When viewed separately, DEL data and docked poses provide noisy signals of binding affinity, but our DEL-Dock model effectively combines these two modalities to better extract the learning signals from the data.
In this work, we engage the commonly studied protein human carbonic anhydrase (CAIX) by training our model on publicly available DEL data generated by Gerry et al. 12 . On a held-out evaluation data set of 3041 molecules with experimental affinity measurements extracted from the BindingDB 24 web database, we demonstrate our approach effectively learns enrichment scores that are well correlated with binding affinity. Moreover, we show that our model is capable of extracting insights into the determinants of docked proteinligand complexes that are most influential for protein binding.

Methods
Our model combines two different representation modalities, molecule-level descriptors and docked protein-ligand complexes, to capture the latent aspects of protein binding from a probabilistic perspective (Fig. 2). The combinatorial construction of DELs motivates using expressive molecular representations to capture statistical correlation between the building block substructures used in DEL synthesis. We use Morgan fingerprint, calculated with RDKit (version 2020.09.1), 25 as basis for our molecular representations, which is a standard descriptor for representational problems on small molecules 26 and also provide the added benefit of their simple construction and rapid processing. Morgan fingerprints compute a structural bit hash of the molecule by enumerating k-hop substructures about each atom.
Since there are many shared structural features across different molecular compounds, these fingerprints constitute a simple representation that has demonstrated remarkable empirical performance throughout cheminformatic domains. We represent docked protein-ligand poses using a pretrained voxel-based CNN model from GNINA, which captures spatial relationships by discretizing space into three-dimentional voxels and leveraging CNNs to learn complex hierarchical representations. [27][28][29] The CNN models used in this work was originally trained on the PDBBind 30 database, capitalizing on this supervised data source to capture the important features that characterize protein-ligand interactions. Let X denote the set of molecules in our data, where each molecule x ∈ X has an associated set of n docked poses {p 1 , p 2 , ..., p n } ∈ P and c matrix i ∈ C matrix , c target i ∈ C target are the ith replicates (repeated experiments) of count data from the beads-only control and target protein experiments respectively. Additionally, we can define the following featurization transformations that are used to construct the molecule and pose embeddings: Φ : X → [0, 1] n φ is the function that generates a n φ -bit molecular fingerprint; here, we use a 2048-bit Morgan fingerprint with radius 3. Ψ : X × P → R n ψ is the transformation that outputs an embedding of the molecule and a specific spatial protein-ligand complex, where we use a pre-trained voxel-based CNN to perform this transformation.
Let h fps = MLP(Φ(x)) be the molecule embedding learned by our model, which is com-  This design choice is motivated by the chance that sparse zero counts in the data could be explained as an artifact of imperfect sequencing technology, and as a result we directly incorporate this assumption into the structure of our model. We note that previous approaches have employed Gamma-Poisson distributions to model DEL count data in the past, 16 but we make the assumption here that replicate data for the same molecule is sampled from an identical distribution.
Let C be distributed as a zero-inflated Poisson (ZIP), with its probability density function (PDF) defined in eq. (2). Here, λ is the rate parameter of the underlying Poisson distribution, and π denotes the occurrence of choosing the zero distribution, and is taken to be the empirical average (when π = 0, this reduces to the typical Poisson distribution). Empirically, we estimate these zero-count probabilities from the zero-count frequencies in the control (π matrix ≈ 0.0075) and protein target (π target ≈ 0.55) experiments. Since we model the control and protein experiments as originating from separate underlying count distributions, we compute two distinct rate parameters for each ZIP distribution as shown in eq. (3).
The observed target counts is a function of both matrix binding and binding to the protein target, so the rate parameter for the target distribution is a function of λ matrix and λ target . We assume an additive functional form for the latent matrix and target enrichments, which we find works well empirically. We note however that there could be more motivated methods to parameterize the interactions of matrix and target binding. The final loss function is then a typical negative log-likelihood (NLL) loss over the observed counts from both the control and target experiments eq. (5).

DEL Data
To train our model we use publicly available DEL data collected by Gerry et al. 12 16 This data prepossessing provides the interpretation of each molecule count as a molecular frequency of that molecule within the DEL library. The processed data set is then used to train our models employing a 80/10/10 -train/validation/test split.
Complete training details are further provided in the Supporting Information.

Evaluation Data
We evaluate the performance of our models on benchmarks using an external set of affinity measurements of small molecule curated from the BindingDB 24 web database. We queried binding affinities for the Human Carbonic anhydrase 9 (CAIX) protein target (UniProt: Q16790), and kept only molecules containing the same atom types as those present in the DEL data set (C, O, N, S, H, I). This external evaluation data set is composed of 3041 small molecules with molecular weights ranging from ∼25 amu to ∼1000 amu and associated experimental inhibitory constant (K i ) measurements ranging from ∼0.15 M to ∼90 pM. We use the median affinity value in the cases where multiple different affinity measurements were reported for the same molecule. We also consider a subset of this dataset which consists of the 521 molecules with molecular weights between 417 amu and 517 amu (Fig. 6 in the Supporting Information). These molecular weights correspond to the interquartile range bounding the 10th and 90th percentiles of the molecular weights in training dataset. This restricted subset presents a more challenging test as differentiation cannot rely on only extensive properties such as molecular weight, but must also effectively identify chemical motifs that impact molecular binding within this tightly bound range of molecular weights.
We notice that simple properties such as molecular weight or benzenesulfonamide presence, which is known to be an important binding motif for carbonic anhydrase, 12,32,33 achieve better baseline performance on the full evaluation data compared to the restricted subset.
These metrics suggest that this subset is more challenging as predictors must learn beyond these simple molecular properties to achieve good performance.

Docking
We perform molecular docking to generate a collection of ligand-bound poses to a target protein of interest for all molecules within in our training and evaluation data sets. Docking

Results
We demonstrate that our model outperforms previous systems on DEL enrichment prediction by jointly combining topological features from the molecular graph and the spatial 3-D protein-ligand information, and additionally illustrate the capability of our model to better rank ligand poses compared to traditional docking. Our model learns latent binding affinity for each molecule to both the matrix and the target as the denoised signals compared to the observed count data. In this interpretation we should expect higher enrichment scores predicted by our model to be well-correlated with binding affinity, and therefore provide a useful metric for predicting anticipated protein binding in virtual screening campaigns.

DEL-Dock outperforms baselines using only docking or moleculelevel descriptors
To evaluate the performance of our model, especially with respect to out-of-domain protein binding prediction, we first train our model on DEL data screened against the human carbonic anhydrase IX (CAIX) protein target, 12 and then predict enrichment scores for molecules with externally measured experimental binding affinities to CAIX. We evaluate performance in this setting by measuring spearman rank-correlation coefficients between predicted enrichments and the experimental affinity measurements, which is a metric that is agnostic to the scale of the values. Our model only restricts the enrichment scores to be positive quantities, with no specific distributional constraints, so spearman rank-correlation, which computes a correlation based only on the ordinal ranking of the predicted enrichments, is well suited for our test scenario.
Our method, DEL-Dock, which combines information from docked complexes with molecular descriptors outperforms previous techniques which only utilize one of these two data modalities (see Table 1). We find that traditional docking scores alone generated from AutoDock Vina 34 result in the worst overall correlations, commensurate with previous ob-servations that docking scores alone are typically not reliable predictors of binding affinity. 19 Performance based on docked poses alone is however greatly improved when re-scoring the docked poses using pretrained GNINA 36 CNN models. Another set of baselines we consider are DEL models that rely only on molecular descriptors. First, we consider a simple model that involves training a random forest (RF) on the Morgan fingerprints using the enrichment metrics originally formulated to facilitate analysis of the DEL data set by Gerry et al. 12 .
We then similarly predict the enrichment scores of molecules in the held-out dataset and the correlation with experimental Ki data. This baseline achieves reasonable performance on both the full and subset evaluation data, especially given the simplicity of the model. DEL-Dock better ranks molecules with known binding chemical motifs While our approach displays good prediction accuracy with respect to experimental binding measurements, we also find our model provides insights into the structural and chemical factors that influence binding. Compounds containing benzenesulfonamide have been well established in literature as the primarily chemical motif that drives small molecule binding to carbonic anhydrase. 12,32,33 Though we do not explicitly incorporate this as a learning signal for our model, we observe that our model is able to learn this association, visibly predicting sulfonamides within our evaluation data set as more highly enriched compared molecules which do not contain benzenesulfonamides (Fig. 3a). Interestingly, we observe a comparatively large fraction of non-benzenesulfonamides identified as good binders with

DEL-Dock offers interpretability through its attention mechanisms
Lastly, we further demonstrate the interpretability of or model by examining the distribution of attention scores learned by our model for a specific molecule (Fig. 4). For this molecule, only 7 out of 20 docked poses correctly coordinate the sulfonamide group with the zinc ion buried in the protein active site. Our model appropriately identifies this binding mode and learns attention scores that more favorably rank these 7 correctly coordinated poses (Fig. 4).

Conclusions
In this work we present an approach for modeling DEL data that combines docking-based and molecular descriptor-based data modalities. Our approach involves predicting two interleaved quantities, enrichment scores, that explain the sequencing counts of the panning experiment measurements for both the on-target protein and the off-target control beads.
We evaluate our method by first training on DEL data screened against the human car- or GNINA, [27][28][29]36 our model more reliably selects poses displaying the appropriate zincsulfonamide coordination-which is known to be the predominant binding mode for carbonic anhydrase. 12,32,33 Our model is interestingly capable of learning good pose selection in an unsupervised manner, training only on the voluminous DEL data rather than requiring crystal structures to serve as the source of supervision.
There are some limitations to our approach, however. Our approach assumes that we can obtain reasonably accurate protein-ligand docking poses, which is not always true. We can only utilize this model for proteins which have 3-D crystal structures; and even if protein crystal structures are available, they may not always be accurate. Additionally, we are limited by the quality of the docking software, which may fail to capture the actual correct binding modes, or ranking the good binding modes highly.
Our work focuses on introducing the concept of utilizing docked poses to improve DEL models, but there are several avenues for future work. While we use Morgan fingerprints as our molecule featurizer, deep learning approaches have the potential to generate more expressive representations, such as Graph Neural Networks (GNNs) that have demonstrated excellent predictive performance on many molecular property prediction tasks. 37 In our application, equivariant GNNs could also be used as an alternative featurizer to CNNs for embedding docked poses, which would provide the added benefit of producing explicitly roto-translationally symmetric representations. 38 We only select the top ranking poses from docking, but providing a more diverse set of poses could be more useful instead. Future research directions could also leverage our approach to unsupervised pose selection for downstream free energy calculations in larger-scale virtual screening campaigns. Our approach paves the way for multi-modality modeling of DEL data and unsupervised learning of bespoke, DEL-conditioned, scoring functions.

Supporting Information
Training settings Featurizations for the docked poses are generated using pre-trained GNINA models provided in gnina-torch. 28,29 The dense variant of the GNINA models composed of densely connected 3D residual CNN blocks introduced in Francoeur et al. 36

Ablations
We present here a number of ablations on our model, exploring some different architectural components and design choices (Table 2). First, we see that training models with only fingerprint representations, without incorporating any information from the docked poses, results in a marked decrease in performance. On the other hand models, trained using only CNN representations perform much better and display comparable performance to only GNINA pre-trained models (Table 1). This represents an intuitive result, as this training setting is effectively equivalent to fine-tuning GNINA using a multi-instance learning over the pose representations. Interestingly, we find that training on the CNN features alone already achieves good binding pose selection based on the latent attention scores, a feature our model is evidently capable of learning in isolation of the fingerprint representations. Also shown as a baseline is the MLP network trained using the bespoke loss function for modelling DEL data presented by Lim et al. 9 . This approach represents the lower performing of the two architectures explored by Lim et al. 9 , the other being the GNN architecture presented as a baseline in the main text (Table 1). Interestingly using a zero-inflated loss appears to be critical to our performance, resulting in ∼25% increase in spearman rank-correlation on the full evaluation set and greater than a 2× increase on the subset. We suspect this performance jump could be related to the disparity in zero-counts between the control and on-target experiment: the control experiments have a zero-count frequency of ∼0.75% while the protein target experiments have a zero-count frequency of ∼55%. Using a zero-inflated distribution could provide our model more flexibility to explain zero-counts as an artifact of the data generation process, rather than an outcome of poor protein binding.
Instead of using pre-computed CNN features from GNINA we also explored training our model directly from the voxel representations using frozen CNN featurizers. The benefit of this approach is the ability to use data augmentation via radom rotations and translations to implicitly enforce that the learned CNN embeddings remain roto-translationally equivariant.
While we notice performance on the evaluation subset is comparable with our trained on pre-computed CNN features (Table 1), the performance on the full data set is slightly reduced. We suspect this result could be due to the computational challenges of using voxelized representations. In particular, when training over many docked poses (in our case 20 poses per molecule) our batch size is effectively 20× larger -which presents a significant memory bottleneck as the voxel representation requires storing a 48×48×48×28 molecular grids (three dimensions discretizing space, and one for different atom types). Furthermore, our pre-computed features are already being generated with pre-trained CNN featurizers that have been trained using data augmentation, albeit on PDBBind for the separate task of affinity and pose prediction. Nevertheless, we certainly expect improved performances could still be achieved for these full differentiable training approaches given the appropriate compute resources and further tuning.
Lastly, presented in Table 3

Supplementary Figures
Shown in Fig. 5 is a TSNE embedding of the DEL data set alongside our evaluation data.
This TSNE embedding is generated by representing each molecule with a concatenation of three fingerprint representations: a 2048-dimensional Morgan fingerprint with a radius of 3, 167-dimensional MACCS (Molecular ACCess System) fingerprint, and finally a 2048dimensional atom pair fingerprint. All fingerprints are calculated using RDKit. Sci-kit learn is then used to generate the TSNE embedding using a tanimoto similarity metric with a perplexity of 30 trained on the combined DEL and evaluation data. We notice the evaluation data is largely isolated from the DEL data in this TSNE embedding, serving as an indication that our evaluation data is markedly different, or out of domain, than the DEL data used in training our models. Fig. 7