{'fraction_non_alphanumeric': 0.03881345446257042, 'fraction_numerical': 0.011722184233662206, 'mean_word_length': 4.76967875258313, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 5, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Learning Graph Normalization for Graph Neural Networks

Graph Neural Networks (GNNs) have attracted considerable attention and have emerged as a new promising paradigm to process graph-structured data. GNNs are usually stacked to multiple layers and the node representations in each layer are computed through propagating and aggregating the neighboring node features with respect to the graph. By stacking to multiple layers, GNNs are able to capture the long-range dependencies among the data on the graph and thus bring performance improvements. To train a GNN with multiple layers effectively, some normalization techniques (e.g., node-wise normalization, batch-wise normalization) are necessary. However, the normalization techniques for GNNs are highly task-relevant and different application tasks prefer to different normalization techniques, which is hard to know in advance. To tackle this deficiency, in this paper, we propose to learn graph normalization by optimizing a weighted combination of normalization techniques at four different levels, including node-wise normalization, adjacency-wise normalization, graph-wise normalization, and batch-wise normalization, in which the adjacency-wise normalization and the graph-wise normalization are newly proposed in this paper to take into account the local structure and the global structure on the graph, respectively. By learning the optimal weights, we are able to automatically select a single best or a best combination of multiple normalizations for a specific task. We conduct extensive experiments on benchmark datasets for different tasks, including node classification, link prediction, graph classification and graph regression, and confirm that the learned graph normalization leads to competitive results and that the learned weights suggest the appropriate normalization techniques for the specific task. Source code is released here https://github.com/cyh1112/GraphNormalization.


INTRODUCTION
Graph Neural Networks (GNNs) have shown great popularity due to their efciency in learning on graphs for various application areas, such as natural language processing Liu et al., 2019a;Zhang et al., 2018), computer vision (Wang et al., 2018;Li et al., 2020;Cheng et al., 2020), point cloud (Shi & Rajkumar, 2020;Liu et al., 2019b), drug discovery (Lim et al., 2019), citation networks (Kipf & Welling, 2016), social networks (Chen et al., 2018) and recommendation (Fan et al., 2019;Wu et al., 2019). A graph consists of nodes and edges, where nodes represent individual objects and edges represent relationships among those objects. In the GNN framework, the node or We also extend these four normalization methods to edge features as shown in (e), (f), (g), and (h).
edge representations are alternately updated by propagating information along the edges of a graph via non-linear transformation and aggregation functions (Wu et al., 2020;Zhang et al., 2018). GNN captures long-range node dependencies via stacking multiple message-passing layers, allowing the information to propagate for multiple-hops . Then, the node or edge representations can be used for downstream tasks such as node classification, link prediction, and graph regression and classification.
In essence, GNN is a new kind of neural networks which exploits neural network operations over graph structure. Among the numerous kinds of GNNs (Bruna et al., 2014;Defferrard et al., 2016;Maron et al., 2019;Xu et al., 2019), message-passing GNNs (Scarselli et al., 2009;Li et al., 2016;Kipf & Welling, 2016;Velickovic et al., 2018;Bresson & Laurent, 2017) have been the most widely used due to their ability to leverage the basic building blocks of deep learning such as batching, normalization and residual connections. To update the feature representation of a node, Kipf & Welling (2016) proposed a Graph ConvNets (GCN) to employ an averaging operation over the neighborhood node with the same weight value for each of its neighbors. GraphSage (Hamilton et al., 2017) samples a fixed-size neighborhood of each node and performs mean aggregator or LSTM-based aggregator over the neighbors. In Graph Attention Networks (GAT) (Velickovic et al., 2018), an attention mechanism is incorporated into the propagation step. It updates the feature representation of each code via a weighted sum of adjacent node representations. Monti et al. (2017) has present MoNet which designs a Gaussian kernel with learnable parameters to assign different weights to neighbors. GatedGCN (Bresson & Laurent, 2017) has achieved state-of-art results on many datasets (Dwivedi et al., 2020). GatedGCN explicitly introduces edge features at each layer and updates edge features by considering the feature representations of these two connected nodes of the edge. GatedGCN designs edge gates to assign different weights to different elements of each neighboring node representation. In addition, GatecGCN uses residual shortcut, batch normalization, and activation function to update the node representations. More details about GNNs are provided in Appendix A.
It is well accepted that normalization techniques (Ioffe & Szegedy, 2015;Ba et al., 2016;Wu & He, 2018;Ulyanov et al., 2016) are the critical ingredients to effectively train deep neural networks. Batch normalization (BN) (Ioffe & Szegedy, 2015) is widely used in training deep neural networks (He et al., 2016;Silver et al., 2017) to perform global normalization along the batch dimension. Hereafter various normalization methods have been developed from different perspectives. For instance, layer normalization (LN) (Ba et al., 2016) and group normalization (Wu & He, 2018) operate along the channel dimension, and instance normalization (Ulyanov et al., 2016) performs a BN-like normalization but only for a sample. A detailed introduction of BN and LN are provide in the Appendix B. In addition, Switchable Normalization  utilizes three distinct scopes to compute statistics (means and variances) including a channel, a layer, and a minibatch. In graph neural networks, (Dwivedi et al., 2020) utilizes BN for each graph propagation layer to train GNNs. Zhao & Akoglu (2020) introduces a novel normalization layer denoted as PAIRNORM to mitigate the over-smoothing problem, which prevents all node representations from homogenization. PAIRNORM focuses on differentiating the distances between different node pairs. Unfortunately, it ignore the local neighbor structure and global graph structure. Each normalization method has its advantages and is suitable for some particular tasks. For instance, BN has achieved perfect performance in computer vision, however, LN outperforms BN in natural language processing (Vaswani et al., 2017). Moreover, in previous work, only one of the mentioned normalization methods is selected and it is used in all normalization layers of a neural network. This will limit the performance of a neural network and it is hard to decide which normalization method is suitable for a specific task.
In this paper, we investigate four types of graph normalization methods, including node-wise normalization, adjacency-wise normalization, graph-wise normalization and batch-wise normalization, as illustrated in Figure 1 (a)-(d). The node-wise normalization only considers its own features and compute the statistics over the elements of its feature vector. Node-wise normalization is equivalent to layer normalization in nature. According to the adjacent structure, we normalize the node feature only using the mean and variance of its adjacent neighborhoods. We terms this normalization method as adjacency-wise normalization, as shown in Figure 1 (b). We also introduce a graph-wise normalization which normalizes the nodes using the mean and variance computed from each graph alone as shown in Figure 1 (c). BN computes the statistics (mean and variance) over a mini-batch and is named as batch-wise normalization (Figure 1 (d)) in this paper. Moreover, we also extend these normalization methods to handle the edge features as shown in Figure 1 (e)-(h). Then, we propose to learn graph normalization by optimizing a weighted combination of normalization techniques at four different levels, including node-wise normalization, adjacency-wise normalization, graph-wise normalization, and batch-wise normalization. By learning the optimal weights, we are able to automatically select the most suitable normalization or a mixture of multiple normalization methods for a specific task.
The contributions of the paper can be highlighted as follows.
• According to different statistical levels of mean and variance of graph, we formulate the graph normalization methods into four levels: node-wise, adjacency-wise, graph-wise, and batch-wise.
To the best of our knowledge, the adjacency-wise normalization and graph-wise normalization are proposed for training GNNs for the first time.
• We propose a novel framework to learn graph normalization by optimizing a weighted combination of normalization methods at different levels. By learning the optimal weights, we are able to automatically select the best normalization method or the best combination of multiple normalization methods for a specific task. We observe that graph-wise normalization performs well on some node classification tasks and batch-wise normalization yields better performance on graph classification and regression tasks, which is consistent with the distribution of the learned dominant weights in our proposal.
• We conduct extensive experiments on several benchmark datasets with different tasks to quantitatively evaluate the listed four normalization methods and the learned normalization methods in various scenarios.

GRAPH NORMALIZATION
where V k is the set of nodes and E k is the set of edges. GNNs propagate information along the edges E of a graph to update the representation of each node for downstream tasks, such as node classification, link prediction and graph classification. We denote v k,i as the i-th node of the G k and h v k,i ∈ R d as the corresponding node feature. h j v k,i represents the jth element of node feature h v k,i . N(v k,i ) represents the neighbors of node v k,i and v k,i .

NORMALIZATION METHODS FROM DIFFERENT LEVELS
Currently, most GNNs use BN, which computes statistics over a mini-batch, after the messagepassing layer to normalize feature representation. Note that graph data contains rich structural information. In this paper, by considering the structure information at different scales, we present and rewrite several normalization methods for graph-structure data.
Node-wise Normalization. For the node v k,i , its feature h v k,i has d elements, as Figure 1 (a). Each of them may have different distribution and we can normalize them to reduce the "covariate shift" problem. Then, according to layer normalization, h v k,i can be normalized by itself: where µ (n) k,i and σ (n) k,i are the mean and variance along the feature dimension for node v k,i , and 1 ∈ R d represents a d-dimension vector filled with the scale value 1. If we normalize the edge features, each edge feature (Figure 1 (e)) is processed individually as Equation (1). Essentially, node-wise normalization is equivalent to applying LN to each node of a graph.
Adjacent-wise Normalization. Each node in a graph has its own neighbors. However, node-wise normalization performs normalization on each node alone and ignores the local structure of each node. For a node v k,i in a graph G k , we consider its topological structure of its adjacent nodes N(v k,i ), as Figure 1 (b). The number of adjacent nodes |N(v k,i )| varies with the node v k,i . N(v k,i ) may only contain itself. Thus, the statistics (mean and variance) computed over N(v k,i ) are unstable. Like LN, we can compute mean and variance across over all elements of feature vectors corresponding adjacent nodes. So, the adjacency-wise normalization for node v k,i is written as: where µ Graph-wise Normalization. Thirdly, nodes belonging to G k naturally are composed of a group. In order to preserve the global structure of a graph, the feature representation of each node is normalized based on the statistics computed over G k . So, we define graph-wise normalization for node v k,i as the following Equation (3): (3) are the mean and standard deviation vectors of G k . Similarly, for the edges E k of Graph G k (Figure 1 (g)), we compute the mean and variance over all edges of G k . When the task has only one graph, graph-wise normalization is similar to BN while the later uses a smoothing average updater but graph-wise normalization does not.
Batch-wise Normlaization. To keep training stable, BN is one of the most critical components. For a mini-batch, there are N graphs. We can compute the mean and standard deviation across over the graphs of a mini-batch, then a node feature h v k,i is normalized by the following equation, which is batch-wise normalization (GN b ): where n means the total nodes of all N graphs. GN b performs normalization over nodes of all N graphs in a mini-batch and is the same as BN (Ioffe & Szegedy, 2015) in nature.
The properties of four normalization methods are summarized as follows.
• Node-wise normalization is equivalent as LN in operation. Node-wise normalization only considers each node's feature alone, but ignores the adjacent and whole graph structures.
• Adjacent-wise normalization takes the adjacent nodes into account. It reflects the difference between the node and its neighbors.
• Graph-wise normalization takes the features of all nodes in a graph into account. It embodies the difference of all the nodes in the whole graph. In ideal situation, this method can stand out the target node from the graph.
• Batch-wise normalization is the same as the standard batch normalization in nature. It expresses differences among the graphs. When the task only has one graph, then the batch-wise normalization is similar with the graph normalization except that momentum average is used in batch-wise but not in the graph-wise.

LEARNING AN UNITED GRAPH NORMALIZATION
Although we have defined several normalization methods for the graph-structured data, different tasks prefer to different normalization methods and for a specific task, it is hard to decide which normalization method should be used. Moreover, one normalization approach is utilized in all normalization layers of a GNN. This will sacrifŷ where λ n , λ a , λ g , λ b ∈ R d are defined as the trainable gate parameters with the same dimension as h v k,i , γ and β denote the trainable scale and shift, respectively. λ n , λ a , λ g , λ b indicate the contribution of the corresponding normalized feature toĥ v k,i . Thus, we constrain the elements of λ n , λ a , λ g , λ b to be in the range [0, 1] by the following scheme: {λ u } u∈{n,a,g,b} in Equation (5) are important weights used to combine the normalized feature of each normalizer and they satisfy the summation constraint, u∈{n,a,g,b} λ j u = 1, j ∈ {1, 2, ..., d}. Thus, in this mechanism, if a normalizer is better for a specific task, its corresponding weights may be higher than others. In GN, several normalization methods cooperate and compete with each other to improve the performance of GNNs.
Different normalization methods are suitable for different tasks. In GN, the weights λ n , λ a , λ g , λ b are updated adaptively for a specific task. Usually, the best-performing normalization will have largest weight in the learnt weight distribution. Thus, GN can be an effective tactic to select one type of normalization method or a best combination of several normalization approaches for one specific task.

EXPERIMENTS
As Dwivedi et al. (2020), we evaluate GN n , GN a , GN g , GN b , and the united Graph Normlization (GN) under three frameworks including Graph Convolution Network (GCN) (Kipf & Welling, 2016), Graph Attention Network (GAT) (Velickovic et al., 2018) and GatedGCN (Bresson & Laurent, 2017). We also assess the performance of GNNs without normalization layer named as "No Norm". We use the same experimental setup as Dwivedi et al. (2020). The implementation of GCN, GAT and GatedGCN are found in GNN benchmarking framwork 1 , which uses Deep Graph Library  and PyTorch (Paszke et al., 2019).
Experimental datasets consist of three types of tasks including node classification, link prediction, and graph classification/regression. We use all seven datasets from (Dwivedi et al., 2020), which are PATTERN, CLUSTER, SROIE, TSP, COLLAB, MNIST, CIFAR10, and ZINC. In addition, we apply GatedGCN for key information extraction problem and evaluate the affect of different normalization methods on SROIE , which is used for extracting key information from receipt in ICDAR 2019 Challenge (task 3). For more information about the detailed statistics of the datasets refer to Appendix C.1.
The hyper-parameters and optimizers of the models and the details of the experimental settings are the same as GNN bechmarking framework Dwivedi et al. (2020). To evaluate the performance of GNNs with different depths, we run experiments on CLUSTER and PATTERN datasets with depth {4, 16} respectively. For the other datasets, we fix the number of GCN layer to 4.

NODE CLASSIFICATION
For node classification, its goal is to assign each node v ∈ V to a number of classes. Hence, GNNs predict the label of each node by passing the node representation vector to a MLP. We evaluate the performance of different normalization approaches on CLUSTER and PATTERN datasets. Moreover, we apply node classification to key information extraction. SROIE consists of 626 receipts for training and 347 receipts for testing. Each image in the SROIE dataset is annotated with text bounding boxes (bbox) and the transcript of each text bbox. There are four entities to extract (Company, Date, Address and Total) from a receipt, as shown in Appendix C.2.

CLUSTER AND PATTERN DATASETS
For CLUSTER and PATTREN datasets, the average node-level accuracy weighted with respect to the class sizes is used to evaluate the performance of all models. For each model, we conduct 4 runs with different seeds {41, 95, 35, 12} and compute the average test accuracy as shown in Table 1.
Graph-wise normalization (GN g ) outperforms batch-wise normalization (GN b ) obviousluy in most situations. For instance, when the depth of GNNs is 4, GatedGCN with GN g achieves 9% improvement over GN b on CLUSTER. Batch-wise normalization computes the statistics over a batch data and ignores the differences between different graphs. Different from GN b , GN g performs normalization only for each graph. Thus, GN g can learn the dedicated information of each graph and normalize the feature of each graph to a reasonable range. As we known, the performance of the adjacencywise normalization (GN a ) is similar with that of the node-wise normalization (GN n ). Compared with GN n , GN a consider the neighbors of each node and gets higher accuracies. We can see that GN gets comparable results for different GNNs. The results of GN are close to the best results in most scenarios due to its flexibility and adaptability. GN adaptively learns weight combination of those normalization approaches which better adapt itself to the node classification task.

SROIE DATASET
In this experiment, we test the performance of GatedGCN with several different normalization approaches for extracting key information from a receipt. For a receipt image, each text bounding box is label with five classes, which are Total, Date, Address, Company and Other. Thus, for a receipt, we treat each text bounding box as a node. Feature representation for each node will be supplied by Appendix C.2. "Company" and "Address" usually consist of multiple text bounding boxes (nodes). If and only if all nodes of each entity are classified correctly, this entity is extracted successfully. We compute the mean accuracies for each text field and the average accuracies for each receipt as shown in Table 2.  We can observe that GN g achieves the best performance among all compared normalization methods. In the receipt, there are many nodes with only numeric texts. It is hard to differentiate the "Total" field from other nodes with numeric text. GN g performs well in this field and outperforms the second best by 2.0%. We believe that the graph-wise normalization can make the 'Total' field stand out from the other bounding boxes with numeric text by aggregating the relevant anchor point information from its neighbors and removing the mean number information. Similarly, graph-wise normalization can promote extracting information for the other three key fields. It's interesting that the graph of each receipt is special with Neighboring nodes usually belonging to different classes. Thus, the performance of adjacency-wise normalization is worse than node-wise normalization.

LINK PREDICTION
Link prediction is to predict whether there is a link between two nodes v i and v j in a graph. Two node features of v i and v j , at both ends of edge e i,j , are concatenated to make a prediction. Then the concatenated feature is fed into a MLP for prediction. Experimental results are shown in Table 3.
All the five normalization methods achieve similar performance on TSP dataset. Compared with other normalization methods, the results of GN are very stable. For each GNN, the result of GN is comparable with the best result. In addition, GNNs without normalization layer obtains good performance.
COLAB dataset contains only a graph with 235,868 nodes. When we use adjacency-wise normalization, we encounter out-of-memory problem. Thus, we don't report the results of GN a and GN. Compared with GNNs with normalization layer, the results of GNNs without normalization layer (No Norm) seriously decrease. GN g performs better than GN b . GatedGCN with GN g achieves the best result.

GRAPH CLASSIFICATION AND GRAPH REGRESSION
Graph classification is to assign one label to each graph. We conduce experiments on CIFAR10 and MNIST. Average class accuracies are reported in Table 4. ZINC is a dataset for graph regression.  the mean absolute error (MAE) between the predicted value and the ground truth is calculated for each group. Average MAEs also are reported in Table 4. We can see that GN b outperforms others in most cases. GN g doesn't work well on graph classification and regression. Furthermore, GN g affects the performance of GN. GN integrates the normalized features of GN n , GN a , GN g , and GN b and adptively pays more attention to GN b due to its outstanding performance. Therefore, its performance is comparable with GN b .   Table 5: Performance of different normalization methods on several datasets. For each dataset, we give the performance of two best normalization methods and a new normalization method combined these two best normalizer like Equation (5).
integrates several normalization methods into a framework, achieves competitive results compared with the best single normalization method on various datasets.
In this part, we analyze the effect of each normalization method on different datasets. GN adaptively combines the results of several normalization methods and {λ u } u∈n,a,g,b in Equation (5) indicate the importance weight vector of the corresponding normalizer, respectively. We initialize the important weights {λ u } u∈n,a,g,b in each layer to the equal values (i.e. 1/4). In the training phase, the values of {λ u } u∈n,a,g,b changes between 0 and 1. We investigate the averaged weights in different layers of GatedGCN on several datasets. We get the important weights of each normalizer in each layer from the optimized model. Since λ u ∈ R d , then the averaged weights of each normalizer is calculated over all of the d elements of λ u . According to Figure 2, the weights of each normalizer not only are distinct for different datasets, but also vary for different layers. It implies that distinct layers may have their own preference of normalization methods to achieve good performance. It's interesting that the weights of GN g are larger than others on node classification tasks while GN b is more importance on other tasks. GN has the ability to automatically choose the suitable normalizers for a specific task.
Furthermore, for each dataset, we select the two best normalizers and integrate them into a new normalization method like Equation (5). From Table 5, the combined normalizer achieves the comparable results with the best normalization method for each dataset. Therefore, these results show that the important weights indicate whether the corresponding normalization method is suitable for the current task.

CONCLUSION
In this paper we formulate four graph normalization methods, node-wise, adjacent-wise, graphwise, and graph-wise according to different statistical levels of mean and variance. Node-wise normalization only considers its own statistical information. Adjacent-wise and graph-wise takes local and global graph structures into account. BN computes the statistics in a mini-batch level.
Different normalization methods perform variously in different tasks. We observe graph-wise and adjacent-wise normalizations perform well on some node classification tasks, batch-wise normalization shows better performance on graph classification and regression tasks. Therefore, we propose to learn an effective Graph Normalization (GN) by optimizing a weighted combination of node-wise normalization, adjacency-wise normalization, graph-wise normalization, and batch-wise normalization. Through analyzing the distributions of weights, we can select one or a combinations of several normalization for one specific task.
is its over-smoothing problem. This issue can be partially eased by residual shortcut across layers. One effective approach is to use spatial GNNs, such as GAT (Velickovic et al., 2018) and GatedGCN (Bresson & Laurent, 2017).
GAT (Velickovic et al., 2018) learns to assign different weight to adjacent nodes by adopting attention mechanism. In GAT, the feature representation of v can be updated by: where e u,v measures the contribution of node u's feature to node v: where g(·) is a LeaklyReLU activation function, α is a weight vector and || is the concatenation operation. Similar to Vaswani et al. (2017), to expand GAT's expressive capability and stabilize the learning process, multi-head attention is employed in GAT. GAT has achieved an impressive improvement over GCN on node classification tasks. However, as the number of graph convolutional layers increases, nodes representations will converge to the same value. The over-smoothing problem still exists.
To mitigate the over-smoothing problem, GatedGCN (Bresson & Laurent, 2017) integrates gated mechanism (Hochreiter & Schmidhuber, 1997), batch normalization (Ioffe & Szegedy, 2015), and residual connections (He et al., 2016) into the network design. Unlike GCNs which treats all edges equally, GatedGCN uses an edge gated mechanism to give different weights to different nodes. So, for node v, the formulation of updating the feature representation is: where W , U ∈ R d×d , is the Hadamard product, and the edge gates e vu are defined as: where σ is the sigmoid function, ε is a small fixed constant, A , B , C ∈ R d×d . Different from traditional GNNs, GatedGCN explicitly considers edge featureê v,u at each layer.
where γ and β are trainable scale and shift parameters, respectively. In Equation (13), µ and σ denote the running mean and variance to approximate the mean and the variance of the dataset. So, during testing, they are used for normalization.

B.2 LAYER NORMALIZATION
Layer Normalization (LN) (Ba et al., 2016) is widely adopted in Natural Language Processing, specially Transformer (Vaswani et al., 2017) incorporates LN as a standard normalization scheme. BN computes a mean and a variance over a mini-batch and the stability of training is highly dependent on these two statics. Shen et al. (2020) has showed that transformer with BN leads to poor performance because of the large fluctuations of batch statistics throughout training. Layer normalization computes the mean and variance along the feature dimension for each training case. Different from BN, for each sample h i , LN computes mean µ (L) i and variance σ (L) i across the feature dimension. The normalization equations of LN are as follows: whereĥ i = [ĥ i1 ,ĥ i2 , ...,ĥ i d ] is the feature-normalized response. γ and β are parameters with dimension d.
Overall, there are many normalization approaches (Ulyanov et al., 2016;Wu & He, 2018;Shen et al., 2020;Dimitriou & Arandjelovic, 2020). Shen et al. (2020) has indicated that BN is suitable for computer vision tasks, while LN achieves better results on NLP. For a normalization approach, its performance may vary a lot in different tasks. So, it is very important to investigate the performance of normalization approaches in GNNs.

C DATASETS AND EXPERIMENTAL DETAILS
C.1 DATASET STATISTICS  Table 6: Summary statistics of datasets used in our experiments. The 7th column (AvgEdges) represents the average number of edges per node in a graph. N.C., E.C., G.C., G.R. mean node classification, edge classification, graph classification and graph regression independently.

C.2 SROIE
For a receipt, each text bbox can be viewed as a node of a graph. Its positions, the attributes of bounding box, and the corresponding text are used as the node feature. To describe the relationships among all the text bounding boxes of a receipt, we consider the distance between two nodes v i and v j . If the distance between two nodes is less than a threshold θ, v i and v j are connected by an edge e i,j . The relative positions of two text bounding boxes are important for node classification, hence, we encode the relative coordinates of v i and v j to represent the edge e ij . This information extraction problem can be treated as a node classification problem based on the graph. Our goal is to label each node (text bounding box) with five different classes, including Company, Date, Address, Total and Other. Since GatedGCN explicitly exploit edge features and has achieved state-of-the-art performance on various tasks, for this task, GatedGCN with 8 GCN layers is used. Figure 3: Sample images of the SROIE dataset. Four entities are highlighted in different colors. "Company", "Address", "Date", and "Total" are marked with Red, Blue, Yellow, and Purple individually. The "Company" and the "Address" entities usually consist of several text lines.

D ACKNOWLEDGEMENT
We would like to thank Vijay et al. to release their benchmarking code for our research. We also want to thank the DGL team for their excellent toolbox.