{'fraction_non_alphanumeric': 0.033939662821650396, 'fraction_numerical': 0.009126632019267335, 'mean_word_length': 5.045402298850575, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Graph Rationalization with Environment-based Augmentations

Rationale is defined as a subset of input features that best explains or supports the prediction by machine learning models. Rationale identification has improved the generalizability and interpretability of neural networks on vision and language data. In graph applications such as molecule and polymer property prediction, identifying representative subgraph structures named as graph rationales plays an essential role in the performance of graph neural networks. Existing graph pooling and/or distribution intervention methods suffer from lack of examples to learn to identify optimal graph rationales. In this work, we introduce a new augmentation operation called environment replacement that automatically creates virtual data examples to improve rationale identification. We propose an efficient framework that performs rationale-environment separation and representation learning on the real and augmented examples in latent spaces to avoid the high complexity of explicit graph decoding and encoding. Comparing against recent techniques, experiments on seven molecular and four polymer real datasets demonstrate the effectiveness and efficiency of the proposed augmentation-based graph rationalization framework.


INTRODUCTION
Graph property prediction has attracted attention in different research fields like chemoinformatics and bioinformatics where small molecules are represented as labelled graphs of atoms [7,9,48]. Besides, materials informatics for polymers has emerged in recent years from property prediction to inverse design [4,11]. Polymer are materials consisting of macromolecules, composed of many repeating units. They are ubiquitous in applications ranging from plastic cups and electronics to aerospace structures. New engineering and environmental challenges demand that polymers possess unconventional properties such as high-temperature stability, excellent thermal conductivity, and biodegradability [16,33]. It's important to integrate data science and machine learning into polymer informatics on the tasks of graph classification and regression.
To automate feature extraction from graph data, graph neural network (GNN) models learn node representations through nonlinear functions and layers that aggregate information from node neighborhood [8,12,26,35,42]. Graph pooling is a central component of the GNN architecture that learns a cluster assignment for nodes and passes cluster nodes and their representations to the next layer [14,40]. The final layer returns the representations of entire graphs. Despite the advances of various GNN models, the limitation of data size makes them easily fall into over-fitting and poor generalizability. For example, the number of graphs in molecule benchmark datasets is usually in the range of 1,000 and 10,000; and the size of polymer datasets is even smaller (e.g., ∼600) [17].
Rationalization techniques have been designed to solve the above problem in vision and language data, where the rationale is defined as a subset of input features that best explains or supports the prediction by machine learning models [1,2,23]. However, graph rationalization has not been extensively studied, which aims at identifying representative subgraph structures for accurate and interpretable graph property prediction. Related work mainly focused on advancing graph pooling methods, but cluster assignment could not reflect the most essential part that led to accurate prediction [6,19]. A very recent technique named DIR [34] employed two GNN modules to discover invariant graph rationales: one module separates each input graph into a rationale subgraph and an environment subgraph; the other is a graph property predictor based on the rationale subgraph. As shown at the top in Figure 1 Figure 1: Graph rationalization identifies a rationale subgraph that best explains or supports the prediction of graph property. Our work makes the first attempt to improve graph rationalization by graph data augmentations with environment subgraphs which are the remaining parts after rationale identification. It proposes new augmentation operations, designs and develops a novel graph rationalization framework, and conducts experiments on a large set of molecule and polymer data.
graph , the separator identifies rationale ( ) , and the predictor gives labelˆ( ) based on the rationale. DIR conducted interventions on training distribution to improve the invariance. Unfortunately, when the data size was small, could hardly find good rationales, as reported in our later experiments.
In this work, we make the first attempt to enhance graph rationalization by graph data augmentations. Existing augmentation methods were mainly heuristic modification of graph structure, which could not directly support the identification of graph rationales [22,31,32,46]. We present two augmentation methods based on environment subgraphs that are the remaining parts in the graph after rationale identification. First, rationales are used to train the property predictor, which can be considered as graph examples augmented by environment removal. Second, we replace the environment of input graph with the environment of another graph in the batch: to generate an augmented example: this augmentation method is called environment replacement. The idea is that the rationale can be accurately identified and/or separated from the input graph when the augmented examples are expected to have the same label of the input graph example. Figure 1 presents the idea of generating virtual data for small datasets via data augmentations. Suppose we have rationale ( ) separated from input graph . We use the same GNN-based separator to find environment subgraph ( ) from another graph in the batch. The example augmented by environment replacement is denoted by ( , ) = ( ) ∪ ( ) . The model is trained on this example to predict labelˆ, to be the same as that is the observed label of . We compute two losses on the augmented examples, L and L ("rem" for removal and "rep" for replacement), and jointly optimize and by their combination. The key challenge in the idea implementation is the high computational complexity of decoding for explicit graph forms of rationales, environment subgraphs, and augmented examples, as well as encoding them for representation learning and property prediction. Moreover, it is scientifically and technically difficult to explicitly combine rationale ( ) and environment ( ) from different graphs, as shown in the three augmented examples ( , ) in Figure 1. To address these challenges, we hypothesize that the contextualized representations of nodes play a significant role in rationales, environment subgraphs, and augmented graphs. Thus, we create the representations of all these objects from one latent space.
In this paper, we propose a novel, efficient framework of Graph Rationalization enhanced by Environment-based Augmentations (GREA). It performs rationale-environment separation and representation learning on the real and augmented examples in one latent space to avoid the high complexity of explicit subgraph decoding and encoding. Figure 2 presents the architecture of GREA with a few steps. First, it employs GNN 1 and MLP 1 models to infer the probability of nodes being classified into rationale subgraph m. Second, it employs GNN 2 to create contextualized node representations H. Then, it directly creates the representation vectors of rationales, environment subgraphs and environment-replaced examples, denoted by h ( ) , h ( ) , and h ( , ) , respectively. Note that DIR [34] used a GNN to generate a matrix of masks that indicate the importance of edges and then select the top-edges with the highest masks to construct the rationale. Then it had to run GNNs on all the explicit graph objects. Instead, our GREA uses m and H to compute the representation vectors of the artificial graphs.
We conduct experiments on seven molecule and four polymer datasets. Results demonstrate the advantages of GREA over baselines. For example, it significantly reduces the prediction error on oxygen permeability of polymer membrane with only 595 training examples. The oxygen permeability defines how easily oxygen passes through a particular material. Accurate prediction will speed up material discovery for healthcare and energy utilization.
The main contributions of this work are summarized below: • the first attempt to improve graph rationale identification using data augmentations, including environment replacement, for accurate and interpretable property prediction; • a novel and efficient framework that performs rationaleenvironment separation and representation learning on real and augmented examples in one latent space; • extensive experiments on more than ten molecule and polymer datasets to demonstrate the effectiveness and efficiency of the proposed framework.

RELATED WORK
There are four research topics related to the proposed work. We briefly present their recent studies and compare with ours.
In the GNN models, graph pooling is a central component of their architectures as a cluster assignment function to find local patches in graphs [19]. For example, DiffPool presented a differentiable graph pooling module that learned a differentiable soft cluster assignment for nodes at each layer of a deep GNN, mapped nodes to a set of clusters, and then formed the coarsened input for the next GNN layer [40]. Lee et al. proposed self-attention graph convolution that allows graph pooling to consider both node features and graph topology [14]. Gao and Ji proposed graph pooling and unpooling operations in Graph U-Nets [6]. Xu et al. presented a theoretical framework for analyzing the representational power of GNNs through the graph pooling functions [37]. While graph pooling identifies soft clusters that effectively aggregate information from nodes [39], our work identifies representative subgraph structures for accurate and interpretable predictions of GNN models.

Graph Rationalization
Most rationalization techniques identify the small subset of input features by maximizing the predictive performance based only on the subset itself, called rationale. To rule out spurious correlation between the input features and the output, Chang et al. proposed the concept of invariant rationalization by modeling different environments as non-causal input to train predictors [2]. Rosefeld et al. offered formal guarantees for improvement of the invariant causal prediction on out-of-distribution generalization [1,23].
By introducing causal modeling into GNN optimization, Fan et al. presented a causal representation framework for GNN models to perform on out-of-distribution graphs [5]. Li et al. proposed OOD-GNN that employed a novel nonlinear graph representation decorrelation method that used random Fourier features to encourage GNNs to eliminate the statistical dependence between relevant and irrelevant graph representations [15]. Very recently, Wu et al. proposed the first work called DIR to approach causal rationales for GNNs to improve the interpretability and predictive performance on out-of-distribution data [34]. DIR conducted interventions on the training distribution to create multiple distributions. Unfortunately, distribution intervention might not be the optimal solution to graph rationale identification. Also, the edge selection method suffers from high computational complexity for rationale creation. Moreover, the studies were mainly performed on synthetic data. In this paper, we make the first attempt to define "environment" in graph data, augment data examples by environment replacement, develop an efficient framework, and conduct experiments on a large set of real molecule and polymer data. We find that augmentationenhanced graph rationalization is more effective than DIR.

Graph Data Augmentation
Graph data augmentation (GDA) techniques [3,44,45,47] have improved the performance on semi-supervised node classification, such as DropEdge [22], NodeAug [32], and GAug [46]. Besides, many GDA techniques have been designed for graph-level tasks, aiming at creating new training examples by modifying input graph data examples. For example, GraphCrop regularized GNN models for better generalization by cropping subgraphs or motifs to simulate real-world noise of sub-structure omission [31]. M-Evolve presented two heuristic algorithms including random mapping and motif-similarity mapping to generate weakly labeled data for small datasets [48]. MH-Aug adopted the Metropolis-Hastings algorithm to create augmented graphs from an explicit target distribution for semi-supervised learning [21]. Meanwhile, graph contrastive learning learned unsupervised representations of graphs using graph data augmentations to incorporate various priors [41]. Zhu et al. [49] proposed adaptive augmentation that incorporated various priors for topological and semantic aspects of graphs. Specifically, it designed augmentation schemes based on node centrality measures to highlight important connective structures and corrupted node features by adding noise to unimportant node features. A comprehensive survey of GDA is given by Zhao et al. [44].

Graph Learning on Polymer Data
Material informatics uses machine learning approaches to fast screen material candidates or generate new materials meeting certain criteria, so as to reduce the time of material development. When most related research performed on molecule data [7], polymer researchers have developed a benchmark database and developed machine learning techniques for polymer data, called polymer embeddings [4,11]. They can be used to perform several polymer informatics regression tasks for density, glass transition temperature, melting temperature, and dielectric constants [16,17,33].
Graph " … : Node representations : Node probability of being classified into Rationale

PROBLEM DEFINITION
Graph Property Prediction. Let = (V, E) be a graph of nodes and edges, where V is the set of nodes (e.g., atoms) and E ⊆ V × V is the set of edges (e.g., bonds between atoms). We use ∈ Y to denote the graph-level property of , where Y is the value space. It can have a categorical or numerical value, corresponding to the task of classification or regression, respectively.
A graph property predictor takes a graph as input and predicts its labelˆ. Specifically, a GNN-based predictor employs a GNN encoder to generate node representations H from : where ì ℎ ∈ R is the representation vector of node in graph . GNN encoder GNN(·) can be chosen as GCN [12] or GIN [37].
Once the node representations are ready, a multilayer perceptron (MLP) can project them into a one-dimensional space to obtain a scalar for each node as = MLP( ì ℎ ). As we are more interested in graph-level classification or regression, we first use a readout operator (e.g., average pooling) to get the graph representation h and then apply a MLP to project it to a graph label: Graph Rationalization. Following the existing literature on graph rationalization [5,6,14,34,40] and GNN explanation [39], we use rationale ( ) = (V ( ) , E ( ) ) to indicate the causal subgraph of the property , where ( ) is a subgraph of such that V ( ) ⊆ V and E ( ) ⊆ E. We use ( ) to denote the environment subgraph, which is the complementary subgraph of ( ) in . In contrast with the rationale subgraph ( ) , the environment subgraph ( ) corresponds to the non-causal part of the graph data, which has no causal relationship with the target graph property [2,34].
Let be a GNN-based graph rationalization model that splits an input graph into a rationale subgraph ( ) and an environment subgraph ( ) . Existing graph rationalization methods used only the rationale subgraph as input for property prediction [6,14,34,40]: where (·) = MLP(READOUT(GNN(·))) andˆ( ) denotes the predicted property of the rationale subgraph ( ) . Unfortunately, when suffering from lack of training examples, these methods chose to discard environment subgraphs at the training stage. In the next section, we present a novel framework showing our idea that environment subgraphs can provide natural noise through data augmentation to improve graph rationalization.

PROPOSED FRAMEWORK
In this section, we introduce a novel graph rationalization framework GREA. The key idea is to augment the rationale subgraph by removing its own environment subgraph and/or combining it with different environment subgraphs. Figure 2 shows the overall architecture of GREA: GNN 1 and MLP 1 first separate input graph into rationale subgraph ( ) and environment subgraph ( ) ; GNN 2 next generates node representations H using Eq.

Rationale-Environment Separation
To separate input graph into rationale subgraph ( ) and environment subgraph ( ) , the rationale-environment separator consists of two components: a GNN encoder (GNN 1 ) that generates latent node representations and a MLP decoder (MLP 1 ) that maps the node representations to a mask vector m ∈ (0, 1) on the nodes in the set V. = ( ∈ V ( ) ) is the node-level mask that indicates the probability of node ∈ V being classified into the rationale subgraph. The mask can be on either a node or an edge [34]. we choose to learn masks on the nodes to avoid the computational complexity of edge selection. Hence, m can be calculated as m = (MLP 1 (GNN 1 ( ) where denotes the sigmoid function. Based on m, we have (1 − m) that indicates the probability of nodes being classified into the environment subgraph. GNN 1 and MLP 1 make up the GNN-based graph rationalization model mentioned in Section 3. GREA uses another GNN encoder to generate contextualized node representations H: H = GNN 2 ( ). With m and H, the rationale subgraph and environment subgraph can be easily separated in the latent space. Using sum pooling, we have where 1 denotes the -size column vector with all entries as 1, and h ( ) , h ( ) ∈ R are the representation vectors of graph ( ) and ( ) , respectively.

Environment-based Augmentations
Suppose 1 , 2 , . . . , are the input graphs in one batch for training, where is known as batch size. The rationale-environment separator has generated the graph representations of rationale and environment subgraphs for each graph . That is, we have , h ( ) )}. We design environmentbased augmentations in the latent space of graph representations.

Environment Removal Augmentation.
As graph rationalization aims to find the rationale subgraph which is regarded as the causal factor of graph property, the rationale itself should be good for property prediction. As in the graph pooling methods [6,14] and the graph rationalization as defined in Eq. (3), the environment removal augmentation uses the rationale subgraph only for training the graph property predictor. That is, given the rationale subgraph representation h ( ) of graph , the predicted label iŝ 4.2.2 Environment Replacement Augmentation. As aforementioned in Section 3, the environment subgraphs can be viewed as natural noises on the rationale subgraphs. Hence, in order to enhance the model's robustness against the noise signal brought by the environment subgraphs, for each graph , we combine its rationale subgraph ( ) not only with its own environment subgraph ( ) , but also with all other environment subgraphs ( ) , ∈ {1, 2, . . . , } \ { } in the batch. By replacing the environment subgraph with other environment subgraphs in the batch, the environment replacement augmentation generates − 1 augmented data samples for each graph during training. As the environment replacement happens on the latent space, an aggregation function AGG(·, ·) is used to combine the rationale subgraph representation h ( ) and environment subgraph representation h ( ) . The aggregation function can be any combining/pooling functions such as concatenation, sum pooling, and max pooling. Taking the elementwise sum pooling as an example, the graph representation h ( , ) of a combined graph of rationale subgraph ( ) and environment subgraph ( ) can be calculated as below: For the graph representations h ( , ) generated by the environment replacement augmentation, the MLP property predictor is trained to predict . That is, The graph representations generated by both environment removal augmentation and environment replacement augmentation (i.e., h ( ) and h ( , ) ) are fed into the same property predictor MLP 2 .
The GNN-based property predictor defined in Section 3 includes MLP 2 and GNN 2 that generates the contextualized node representation H.

Optimization.
During training, the type of loss function on the observed graph property ( ) and predicted labels (ˆ( ) and ( , ) ) depends on the type of the property label. For example, when the graph property has binary values in the binary classification task, we use the standard binary cross-entropy loss. When the graph property has real values in the graph regression task, we use the mean squared error (MSE) loss. Without loss of generality, suppose we focus on the binary classification task. Given a batch of graphs 1 , 2 , . . . , , the loss functions for each graph example and its label are defined as where L is the loss for the examples created by environment removal augmentation, and L is the loss for the examples created by the environment replacement augmentation.
Moreover, the following regularization term is used to control the size of the selected rationale subgraph: where ∈ [0, 1] is a hyperparamter to control the expected size of the rationale subgraph ( ) . We penalize the number of nodes in the rationale when it deviates from our expectations.   We use the alternate training schema in Chang et al. [2] to train GREA. That is, we iteratively train (GNN 1 and MLP 1 ) and (GNN 2 and MLP 2 ) for a fixed number of epochs and , respectively. The loss functions for training GREA are where L in Eq. (12) and L in Eq. (13) are used to train (GNN 1 and MLP 1 ) and (GNN 2 and MLP 2 ), respectively. and are hyperparameters that control the weights of L and L , respectively. During inference,ˆ( ) is used as the final predicted property of input graph .

EXPERIMENTS
We conduct experiments to answer the following questions: • Q1) Effectiveness: Does the proposed GREA make more accurate prediction on molecule and polymer properties than existing graph classification/regression methods? • Q2) Ablation study: Do the environment-based augmentations make positive effect on the performance? • Q3) Case study: Based on domain expertise, are the polymer rationale examples identified by GREA representative? • Q4) Efficiency: Does the latent space-based design for augmentations perform faster than explicit graph decoding and encoding? Can we empirically analyze the complexity?
• Q5) Sensitivity analysis: Is the performance of GREA sensitive to hyperparameters such as , , and AGG(·)?

Datasets.
We conduct experiments on four polymer datasets and seven molecule datasets. The statistics of the datasets are given in Table 1, such as number of graphs and average size of graphs. The four datasets GlassTemp, MeltingTemp, PolyDensity, and O 2 Perm are used to predict different properties of polymers such as glass transition temperature ( • C), polymer density g/cm 3 , melting temperature ( • C), and oxygen permeability (Barrer). For all the polymer datasets, we randomly split by 60%/10%/30% for training, validation, and test. Besides polymer datasets, we use seven molecule datasets from the graph property prediction task on Open Graph Benchmark or known as OGBG. For all molecule datasets, we use the scaffold splitting procedure as OGBG adopted [9]. It attempts to separate structurally different molecules into different subsets, which provides a more realistic estimate of model performance in experiments [36]. Dataset descriptions with details are presented in the Appendix A.

Evaluation Metrics.
On the polymer datasets, we perform the tasks of graph regression. We use the coefficient of determination (R 2 ) and Root Mean Square Error (RMSE) as evaluation metrics according to previous works [9,17]. On the molecule datasets, we perform the tasks of graph binary classification using the Area under the ROC curve (AUC) as the metric. To evaluate model efficiency, we use the computational time per training batch (in seconds).

Baseline Methods.
There are three categories of related methods that we can compare GREA with. The first category is graph pooling methods that aim at finding (soft) cluster assignment of nodes towards aggregated representations of graph. They are U-NetsPool [6] and SelfAttnPool [14]. The second category improves the optimization and generalization of learned representations. They include StableGNN [5], OOD-GNN [15], and IRM [1]. The third is DIR for graph rationale identification that was proposed in a very recent work by Wu et al. [34]. To investigate the effect of environment replacement augmentation (denoted by RepAug as a module that may be used or not in the methods), we implement two method variants: (1) DIR+RepAug: We add environment-replaced augmentation to DIR [34] to identify rationales, however, it has to explicitly decode and encode the rationales; (2) GREA−RepAug: We disable the environment replacement augmentation and use only the environment removal augmentation, i.e., rationale subgraphs in GREA. In the experiments, we study two types of GNN models (GCN [12] and GIN [37]) as graph encoders for all the methods. Please refer to Appendix B for details of implementation. Table 2 presents the results on polymer property regression with R 2 and RMSE metrics. Table 3 presents the results on molecule property classification using AUC. Underlined are for the best baseline(s). The best baseline is OOD-GNN for its elimination of the statistical dependence between property-relevant graph representation and property-irrelevant graph representation. The first graph rationalization method DIR was evaluated on synthetic data [34]; unfortunately, it performs poorly on real polymer and molecule (1)

Results on Effectiveness (Q1)
(3) Figure 3: Three polymer examples in O 2 Perm test set to compare graph rationales and property predictions by DIR [34] and our GREA. DIR selects edges to decode rationale subgraphs. Our GREA estimates the probability of nodes being classified into rationales in latent space. The red boxes indicate incoherent edges that DIR selects. The blue boxes indicate coherent node sets that contribute to accurate predictions on oxygen permeability of polymer membrane.
datasets because it selects edges to create rationale subgraphs and thus loses the original contextual information of atoms in the the rationale representations. Compared to them, our GREA with either GCN or GIN consistently achieves the best performance on all the polymer and molecule datasets. On the PolyDensity dataset, GREA with GCN improves R 2 over OOD-GNN relatively by +3.91%. On MeltingTemp, GREA with GIN produces 1.56× R 2 over DIR.

Case Study on Polymer Data (Q3)
Given test polymer examples in the O 2 Perm dataset, we visualize and compare the rationale subgraphs that are identified by from DIR [34] and our GREA in Figure 3. We have three observations. First, the rationales identified by GREA have more coherent structures of atom nodes than those identified by DIR. The red boxes show that quite a few edges in the rationales by DIR are far separated. This is because DIR explicitly decodes the subgraphs by selecting edges. Our GREA estimates the probability of nodes being    included in the rationales and uses the contextualized representations of atoms in the input graphs to create the representations of rationales. So the rationales have coherent structures of nodes. Second, the rationales from GREA are more interpretable and beneficial than the ones from DIR, based on domain expertise in polymer science. Take a look at the first polymer example in Figure 3. The rationale from GREA includes non-aromatic rings and methyl groups. The former group allows larger free volume elements and lower densities (i.e., enlarge microporousity) in the polymer's repeating units, which positively contributes to the gas permeability [24,38]. The latter group is hydrophobic and contributes to steric frustration between polymer chains [38], inducing a positive correlation to the permeability. On the other hand, the rationale from DIR would make property predictor overestimate the oxygen permeability, because it suggests that the double-bonded oxygens, ethers, and nitrogen atoms are positively correlated with the property. However, it conflicts with observations and conclusions from chemical experiments in previous literature [38] where researchers argue that the double-bonded oxygens, ethers, and nitrogen atoms are negatively correlated with gas permeability. For the second and  Figure 6: On four polymer datasets, the performance of GREA (in R 2 ) is not sensitive to hyperparameters and in Eq. (13).
third examples, DIR also predicts through double-bonded oxygens, ethers, and nitrogen atoms, and it overestimates the permeability. Our GREA realizes and employs the true relationship between the functional groups and property and successfully suppresses the representations of non-aromatic rings and methyl groups in the prediction. GREA intrinsically discovers correct relationships between rationale subgraphs and the property. Third, the rationales from GREA are commonly observed across different polymers. We expect rationales to have universal indication on the polymer properties. The rationales identified in the second and third examples both have the fused heterocyclic rings (at the right end of the monomers and highlighted by blue boxes).

Results on Efficiency (Q4)
We conduct efficiency analysis using the ogbg-HIV dataset without losing the generality. Results are presented in Figure 4. When batch size increases, in other words, when a batch has more and more graphs, the time cost per batch of DIR increases significantly; our proposed GREA spends much less time than DIR. Empirically we show that our GREA is more efficient than DIR. This is because GREA does not explicitly decode or encode the subgraphs but directly creates their representations in latent space. Figure 4(b) shows that compared to three most competitive baselines, GREA delivers the highest AUC by learning augmented examples, while spending comparable amount of time.

Sensitivity Analysis (Q5)
Without losing the generality, we conduct three series of sensitivity analyses. First, Figure 6 shows that on four polymer datasets, the performance of GREA in terms of R 2 is insensitive to the hyperparameters and in Eq. (13). Second, Figure 5 shows that the performance is insensitive to rationale size in Eq. (11). Third, on two polymer datasets and one of the most popular molecule datasets, Table 4 compares the effects of different choices of AGG(·) function that aggregates the representations of rationale and environment subgraphs. Sum pooling is generally the best choice.

CONCLUSIONS
In this work, we made the first attempt to improve graph rationale identification using data augmentations, including environment replacement, for accurate and interpretable graph property prediction. We proposed an efficient framework that performs rationaleenvironment separation and representation learning on real and augmented examples in one latent space. Experiments on molecule and polymer datasets demonstrated its effectiveness and efficiency.