Compressed graph representation for scalable molecular graph generation
Recently, deep learning has been successfully applied to molecular graph generation. Nevertheless, mitigating the computational complexity, which increases with the number of nodes in a graph, has been a major challenge. This has hindered the application of deep learning-based molecular graph generation to large molecules with many heavy atoms. In this study, we present a molecular graph compression method to alleviate the complexity while maintaining the capability of generating chemically valid and diverse molecular graphs. We designate six small substructural patterns that are prevalent between two atoms in real-world molecules. These relevant substructures in a molecular graph are then converted to edges by regarding them as additional edge features along with the bond types. This reduces the number of nodes significantly without any information loss. Consequently, a generative model can be constructed in a more efficient and scalable manner with large molecules on a compressed graph representation. We demonstrate the effectiveness of the proposed method for molecules with up to 88 heavy atoms using the GuacaMol benchmark.
Introduction<!>Molecular graph compression<!><!>Molecular graph compression<!><!>Molecular graph compression<!>Learning on graph representation<!><!>Learning on graph representation<!>Molecular graph generation<!>GuacaMol benchmark<!><!>Implementation<!><!>Molecular graph compression<!><!>Molecular graph generation<!>Conclusion<!>
<p>Deep learning has revolutionized the design of novel molecules required for real-world industrial applications. Whereas traditional approaches have mostly been based on human knowledge and intuition, the use of deep learning has enabled the autonomous design of molecules by learning from previously accumulated data [1–3]. Most existing methods use deep generative models, such as variational autoencoders (VAEs) and generative adversarial networks (GANs). Their capabilities depend on the way of representing a molecule. Such representations include simplified molecular-input line-entry system (SMILES) and molecular graph representation. Although the SMILES representation has been demonstrated to be useful, recent research tends to employ the molecular graph representation, which is a natural and intuitive way of representing a molecule by regarding its atoms and bonds as nodes and edges, respectively [1].</p><p>A major challenge for molecular graph generation is addressing the scalability issue caused by its high computational complexity [4]. The representation of a molecular graph \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\mathcal {G}=(\mathcal {V},\mathcal {E})$$\end{document}G=(V,E) on which a model learns, where \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\mathcal {V}$$\end{document}V and \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\mathcal {E}$$\end{document}E are the set of nodes and edges in \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\mathcal {G}$$\end{document}G, typically involves an adjacency expression between its nodes, yielding \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\mathcal {O}(|\mathcal {V}|^2)$$\end{document}O(|V|2) complexity. A naïve approach is to regard only heavy atoms in a molecule as nodes in the corresponding graph representation by treating hydrogen atoms implicitly as node features. This approach is however not scalable for large molecules with many heavy atoms, which are abundant in the real world [5, 6]. Consequently, existing methods were evaluated by limiting the size of the molecules in the training dataset, which was often set to less than 50 heavy atoms. The benchmark datasets with small molecules, such as QM9 [7, 8] and ZINC [9], have been commonly employed in the literature.</p><p>For scalable molecular graph generation, there have been research attempts to alleviate the complexity \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\mathcal {O}(|\mathcal {V}|^2)$$\end{document}O(|V|2) via representational simplification. One approach involves representing a molecular graph as a sequence of vectors and then building an autoregressive model on the sequence representation for the sequential generation of nodes and edges that form a graph. You et al. presented GraphRNN which constructs a model on a node-level sequence representation with M-dimensional adjacency vectors, where M is set to less than \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$|\mathcal {V}|$$\end{document}|V|, by employing breadth-first-search node ordering with which the complexity is reduced to \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\mathcal {O}(|\mathcal {V}|M)$$\end{document}O(|V|M) [10]. Goyal et al. presented GraphGen which transforms a molecular graph into an edge-level sequence based on minimum depth-first-search coding, which leads to a complexity of \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\mathcal {O}(|\mathcal {E}|)$$\end{document}O(|E|) [4]. However, as in the SMILES representation, the sequential nature imposes constraints on the model architecture and prevents the model from capturing molecular similarity and retaining chemical validity. Another approach is to reduce the number of nodes \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$|\mathcal {V}|$$\end{document}|V| directly in the representation. Jin et al. presented junction tree VAE (JTVAE) which represents a molecular graph as a junction tree, whose nodes correspond to valid chemical substructures, using tree decomposition [11]. The compressed representation can be generally applicable to any model architecture. Nevertheless, JTVAE can suffer from high dimensionality due to the dramatic increase in the number of node features, because of the large variety of chemical substructures that appear in the dataset.</p><p>For a more practical application of molecular graph generation, we focus on the latter approach which involves reducing the number of nodes directly in the representation. This study aims to improve the scalability of molecular graph generation to large molecules while maintaining the capability of generating chemically valid and diverse molecular graphs. We present a novel method for the compression of molecular graph representation for scalable molecular graph generation. We designate six small substructural patterns that commonly appear between two heavy atoms in practice and regard their appearances as additional edge features along with the bond types. A molecular graph is compressed by substituting the relevant substructures with new edges. This compression reduces the number of nodes without drastically increasing the number of edge features, making it scalable to large molecules. In addition, the compressed graph can be reconstructed into the original graph without any information loss.</p><!><p>The conventional graph representation of a molecule is an undirected graph whose nodes and edges correspond to heavy atoms and their bonds in the molecule, respectively. Hydrogen atoms are treated implicitly as node features, and thus, they are not regarded as explicit nodes. Formally, a molecular graph is defined as \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\mathcal {G}=(\mathcal {V},\mathcal {E})$$\end{document}G=(V,E), where \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\mathcal {V}$$\end{document}V and \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\mathcal {E}$$\end{document}E denote the sets of nodes and edges, respectively. Each node corresponding to the i-th heavy atom is represented by a node vector \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\mathbf {v}^i \in \mathcal {V}$$\end{document}vi∈V with the dimensionality of p, whose features indicate the atom type, formal charge, and valence information. An edge corresponding to the connection between the i-th and j-th atoms is represented by an edge vector \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\mathbf {e}^{i,j} \in \mathcal {E}$$\end{document}ei,j∈E with a dimensionality of q, whose features are associated with a bond type. The property vector \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\mathbf {y}=(y^1,\ldots ,y^{l})$$\end{document}y=(y1,…,yl) represents the properties of the molecule.</p><!><p>Substructural patterns that commonly appear between two atoms in molecules</p><!><p>Formally, we define a compression function \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\Phi $$\end{document}Φ that compresses an input graph. For an original graph \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\mathcal {G}$$\end{document}G, the corresponding compressed graph \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\mathcal {G}'$$\end{document}G′ is obtained using the function \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\Phi $$\end{document}Φ as1\documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$egin{aligned} \mathcal {G}'=\Phi {(\mathcal {G})}. \end{aligned}$$\end{document}G′=Φ(G).Given the input graph \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\mathcal {G}$$\end{document}G, the function \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\Phi $$\end{document}Φ finds the substructures that are relevant to the six designated patterns. With canonical ordering of the atoms in \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\mathcal {G}$$\end{document}G, each substructure is sequentially converted to an edge by representing its appearance using the corresponding edge feature. The canonical numbers of atoms are used to prioritize which substructure is converted first. When multiple substructures overlap, the one with non-overlapping atoms having smaller canonical numbers is chosen to be replaced by an edge.</p><p>With the addition of edge features, the edge vector of compressed graph \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\mathcal {G}'$$\end{document}G′ has higher dimensionality than that of the original graph \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\mathcal {G}$$\end{document}G. This compression reduces one or two nodes per substructure. There may exist multiple substructures in between an atom pair, and a larger molecule may contain more relevant substructures. A graph will be further compressed if more of the substructural patterns exist in it.</p><!><p>Example of compressed graph representation</p><!><p>The main advantages of compressed graph representation are as follows. Firstly, the compressed representation reduces the number of nodes (i.e., \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$|\mathcal {V}'| \le |\mathcal {V}|$$\end{document}|V′|≤|V|), thereby providing better scalability to large molecules. Secondly, the compression is reversible, meaning that the compressed graph can be reconstructed into the original one without any information loss using a decompression function \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\Phi ^{-1}$$\end{document}Φ-1 (i.e., \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\mathcal {G}=\Phi ^{-1}{(\Phi {(\mathcal {G})})}$$\end{document}G=Φ-1(Φ(G))). Thirdly, it does not drastically increase the dimensionality of edge vectors because only pre-chosen substructural patterns are additionally involved as edge features in the compressed representation (i.e., \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$q'-q$$\end{document}q′-q is a small constant). The increase in edge dimensionality does not significantly affect the scalability.</p><!><p>In this study, we build a non-autoregressive graph VAE (NAGVAE), presented in [12], on the compressed graph representation. The model seeks to find the generative distribution \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$p_	heta (\mathcal {G}|\mathbf {z},\mathbf {y})$$\end{document}pθ(G|z,y) parameterized by \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$	heta $$\end{document}θ. The prior distributions \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$p(\mathbf {z})$$\end{document}p(z) and \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$p(\mathbf {y})$$\end{document}p(y) are set to \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\mathcal {N}(\mathbf {z}|\mathbf {0},\mathbf {I})$$\end{document}N(z|0,I) and \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\mathcal {N}(\mathbf {y}|arvec{\mu }_\mathbf {y},arvec{\Sigma }_\mathbf {y})$$\end{document}N(y|Î¼y,Î£y), respectively. We introduce an approximate posterior distribution \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$q_\phi (\mathbf {z}|\mathcal {G},\mathbf {y})=\mathcal {N}(\mathbf {z}|{arvec{\mu }_\mathbf {z}(\mathcal {G},\mathbf {y}), 	ext {diag}(arvec{\sigma }^2_\mathbf {z}(\mathcal {G},\mathbf {y})}))$$\end{document}qϕ(z|G,y)=N(z|μz(G,y),diag(σz2(G,y))) parameterized by \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\phi $$\end{document}ϕ to address the intractability of the posterior distribution \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$p_	heta (\mathbf {z}|\mathcal {G},\mathbf {y})$$\end{document}pθ(z|G,y).</p><!><p>Schematic diagram of model architecture</p><!><p>Given N molecules and their properties, we form a training dataset \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\mathcal {D}=\{\mathcal {G}_t',\mathbf {y}_t\}_{t=1}^N$$\end{document}D={Gt′,yt}t=1N with the compressed representation, where \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\mathcal {G}_t'=\Phi {(\mathcal {G}_t)}$$\end{document}Gt′=Φ(Gt). Then, the model is trained using the dataset. The objective function for thie training involves the original learning objective of the VAE as well as approximate graph matching, reinforcement learning, and auxiliary property prediction. The details of the model are described in [12].</p><p>The training involves the processing of a graph \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\mathcal {G}$$\end{document}G in the form of a pair \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$(\mathbf {V}, \mathbf {E})$$\end{document}(V,E) comprising a node matrix \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\mathbf {V} \in \mathbb {R}^{|\mathcal {V}| 	imes p}$$\end{document}V∈R|V|×p, where \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\mathbf {V}_i \in \mathbb {R}^{p}$$\end{document}Vi∈Rp is the node vector \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\mathbf {v}^i \in \mathcal {V}$$\end{document}vi∈V, and an edge tensor \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\mathbf {E}\in \mathbb {R}^{|\mathcal {V}| 	imes |\mathcal {V}| 	imes q}$$\end{document}E∈R|V|×|V|×q, where \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\mathbf {E}_{i,j}\in \mathbb {R}^{q}$$\end{document}Ei,j∈Rq is the edge vector \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\mathbf {e}^{i,j} \in \mathcal {E}$$\end{document}ei,j∈E if it corresponds to a bond or substructure and is a zero vector otherwise. This leads to the computational complexity of \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$O(|\mathcal {V}|^2)$$\end{document}O(|V|2). Because the use of the compressed graph representation directly reduces \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$|\mathcal {V}|$$\end{document}|V|, the model becomes more scalable to large molecules.</p><!><p>After training the model, the decoder part \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$p_	heta (\mathcal {G}|\mathbf {z},\mathbf {y})$$\end{document}pθ(G|z,y) is used to generate new molecular graphs. To generate a molecular graph, we sample \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\mathbf {z}_*$$\end{document}z∗ and \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\mathbf {y}_*$$\end{document}y∗ from their prior distributions \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$p(\mathbf {z})$$\end{document}p(z) and \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$p(\mathbf {y})$$\end{document}p(y). They are fed into the decoder to produce a probabilistic output, which is then decoded via node-wise and edge-wise argmax to obtain a compressed graph \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\mathcal {G}'_*$$\end{document}G∗′ as2\documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$egin{aligned} \mathcal {G}'_* =nderset{\mathcal {G}}{	ext {argmax }}p_	heta (\mathcal {G}|\mathbf {z}=\mathbf {z}_*,\mathbf {y}=\mathbf {y}_*). \end{aligned}$$\end{document}G∗′=argmaxGpθ(G|z=z∗,y=y∗).Because \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\mathcal {G}'_*$$\end{document}G∗′ is originally in the form of the compressed representation, we decompress it into its original representation with the decompression function \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\Phi ^{-1}$$\end{document}Φ-1 as3\documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$egin{aligned} \mathcal {G}_* = \Phi ^{-1}(\mathcal {G}'_*). \end{aligned}$$\end{document}G∗=Φ-1(G∗′).The output \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\mathcal {G}_*$$\end{document}G∗ can be interpreted as the chemical structure of a molecule.</p><!><p>We investigated the effectiveness of the proposed method using the GuacaMol distribution-learning benchmark [14]. The training dataset for the benchmark is a standardized subset of the ChEMBL database [6], consisting of 1,591,378 molecules with up to 88 heavy atoms.</p><p>In the benchmark, the performance of a model for generating chemically valid and diverse molecular graphs is evaluated in terms of Validity, Uniqueness, and Novelty of 10,000 molecular graphs generated by the model. Validity is the ratio of valid molecular graphs, for which a molecular graph is counted as valid if it can be processed successfully with RDKit. Uniqueness is the ratio of valid graphs that are not duplicates. Novelty is the ratio of valid graphs that are not present in the training dataset. In addition, Kullback-Leibler Divergence (KLD) and Fréchet ChemNet Distance (FCD) are used to evaluate the success of a model in reproducing the distribution of the training dataset.</p><!><p>Node features of compressed graph representation</p><p>Edge features of compressed graph representation</p><!><p>We employed four SMILES generation models (LSTM [15], VAE [16], AAE [17], and ORGAN [18]) and one molecular graph generation model (GraphMCTS [19]), as implemented in [14], as the baseline models for comparison. SMILES generation models are known to be more scalable to large molecules. The authors of [14] reported that training JTVAE [11] using the GuacaMol benchmark led to an error. We also failed to train the NAGVAE on the original graph representation (NAGVAE\documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$_	ext {original}$$\end{document}original) [12] owing to an out-of-memory error.</p><!><p>Molecular graph compression results on training dataset: a histogram of the number of nodes with the original representation; b histogram of the number of nodes with the compressed representation; c scatterplot between original and compressed representations</p><p>Summary of molecular graph compression results</p><!><p>As evident from the results, the compression function \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\Phi $$\end{document}Φ effectively reduced the number of nodes in the molecular graphs. In particular, molecular graphs tended to be better compressed when the number of nodes was large. The high compression rate contributes to reducing the computational cost and memory usage involved in molecular graph generation.</p><!><p>Molecular graph generation results of baseline and proposed models</p><!><p>From a computational perspective, the use of the compressed representation reduced the computational burden for both the training and inference phases. Considering the complexity \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\mathcal {O}(|\mathcal {V}|^2)$$\end{document}O(|V|2) which increases with the number of nodes, training and inference on a more compact representation with a smaller number of nodes are faster and require lower computational cost and memory usage. This is also evident from the fact that NAGVAE\documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$_	ext {original}$$\end{document}original failed to be trained, whereas NAGVAE\documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$_	ext {compress}$$\end{document}compress was successfully trained with the training dataset. Additionally, the decompression for the compressed graph representation had little effect on the computational burden. The molecular graph generation by NAGVAE\documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$_	ext {compress}$$\end{document}compress, which involves inference with the decoder network \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$p_	heta (\mathcal {G}|\mathbf {z},\mathbf {y})$$\end{document}pθ(G|z,y) and decompression with the function \documentclass[12pt]{minimal}
				sepackage{amsmath}
				sepackage{wasysym} 
				sepackage{amsfonts} 
				sepackage{amssymb} 
				sepackage{amsbsy}
				sepackage{mathrsfs}
				sepackage{upgreek}
				\setlength{\oddsidemargin}{-69pt}
				egin{document}$$\Phi ^{-1}$$\end{document}Φ-1, only took around 0.004 s and 0.001 s per molecular graph on average for the inference and decompression, respectively.</p><p>As demonstrated by the experimental results, the use of compressed graph representation makes molecular graph generation scalable to large molecular graphs without performance degradation with regard to the generation of chemically valid and diverse molecular graphs. We expect that molecular graph compression will shed some light on improving the efficiency and scalability of other molecular graph generation methods without sacrificing their performance.</p><!><p>In this paper, we presented a molecular graph compression method to address the scalability issue of molecular graph generation. We identified six small substructural patterns that commonly appear between atom pairs in real-world molecules. Given a molecular graph, we converted the relevant substructures into new edges by representing them using additional edge features in the compressed graph representation. A generative model was constructed in a more efficient and scalable manner by training the model on the compressed representation. By conducting an experimental investigation using the GuacaMol benchmark, we found that the proposed method reduced the number of nodes significantly without any information loss. The generative model constructed on the compressed representation achieved performance comparable to that of the baseline methods regarding molecular graph generation.</p><p>Although mitigating the high computational complexity intrinsically imposed on molecular graph generation has been challenging, this work successfully demonstrated that the molecular graph compression approach can effectively alleviate the complexity. We expect that this approach will be more effective with the better identification of data-specific substructural patterns that can be regarded as edge features. The use of the compressed representation contributes to a substantial reduction in the computational cost and memory usage, making it scalable to large molecules. This approach can be applied to other molecular graph generation methods to improve their efficiency and scalability, which merits further investigations.</p><!><p>Publisher's Note</p><p>Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.</p>