{'fraction_non_alphanumeric': 0.04788493583184728, 'fraction_numerical': 0.015844710147045925, 'mean_word_length': 4.472404415293553, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 5, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 11, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Signed Bipartite Graph Neural Networks

Signed networks are such social networks having both positive and negative links. A lot of theories and algorithms have been developed to model such networks (e.g., balance theory). However, previous work mainly focuses on the unipartite signed networks where the nodes have the same type. Signed bipartite networks are different from classical signed networks, which contain two different node sets and signed links between two node sets. Signed bipartite networks can be commonly found in many fields including business, politics, and academics, but have been less studied. In this work, we firstly define the signed relationship of the same set of nodes and provide a new perspective for analyzing signed bipartite networks. Then we do some comprehensive analysis of balance theory from two perspectives on several real-world datasets. Specifically, in the peer review dataset, we find that the ratio of balanced isomorphism in signed bipartite networks increased after rebuttal phases. Guided by these two perspectives, we propose a novel Signed Bipartite Graph Neural Networks (SBGNNs) to learn node embeddings for signed bipartite networks. SBGNNs follow most GNNs message-passing scheme, but we design new message functions, aggregation functions, and update functions for signed bipartite networks. We validate the effectiveness of our model on four real-world datasets on Link Sign Prediction task, which is the main machine learning task for signed networks. Experimental results show that our SBGNN model achieves significant improvement compared with strong baseline methods, including feature-based methods and network embedding methods.


INTRODUCTION
According to a well-known Asian proverb, "There are a thousand Hamlets in a thousand people's eyes". It means that different people have different opinions. These differences usually include both positive and negative attitudes. They are reflected in many fields including business, politics, and academics. For instance, a research paper submitted to CIKM may receive completely opposite reviews from two reviewers. Reviewer 1 gives an "Accept" decision, while Reviewer 2 chooses the "Reject" option. Understanding and modeling these differences is a useful perspective on a range of social computing studies (e.g., AI peer review [16] and congressional vote prediction [22]). Figure 1 shows some common application scenarios for signed bipartite networks, including product review, bill vote, and peer review. Some opinions can be viewed as positive relationships, such as favorable reviews on products, supporting the bill, accepting a paper, and so on. Meanwhile, some opinions are negative links that indicate negative reviews, disapproving a bill, rejecting a paper, and so forth. These scenarios can be modeled as signed bipartite networks, which include two sets of nodes (i.e., U and V) and the links with positive and negative relationships between two sets. Compared with unsigned bipartite networks, the links of signed bipartite networks are more complicated, including two opposite relationships (i.e., positive and negative links). Besides, previous works on signed networks only focus on unbipartite signed networks, which are networks that have a single node type [9]. Different node types in signed bipartite networks represent different things. Modeling signed bipartite networks is a promising and challenging research field.
With modeling the above scenarios into signed bipartite networks, we do social network analysis on real-world datasets and use advanced graph representation learning methods to model them. In signed networks, balance analysis is one of the key research problems in signed graph modeling [18]. A common balance analysis method is to count the number of balanced signed triangles in unbipartite signed networks [26]. For signed bipartite networks, [5] defines the signed butterfly isomorphism, and uses it to analyze the balance in signed bipartite networks. But signed butterfly isomorphism may be missing due to data sparsity. In this paper, we offer a new perspective for analyzing balance theory on signed bipartite networks. By sign construction, we construct the links between the nodes in the same set and count the signed triangles for the links between the nodes in the same set. We analyze the balance theory of signed bipartite networks from both two perspectives and explore the balance change in the peer review scenario. We find that after rebuttal, the balance of the review signed bipartite network increased. In addition to social network analysis, graph representation learning is another important tool. Graph Neural Networks (GNNs) have achieved state-of-art results in graph representation learning. Combing two perspectives, we propose a new Signed Bipartite Graph Neural Networks (SBGNNs). This model follows the message passing scheme, but we redesign the message function, aggregation function, and update function. Our SBGNN model achieves the state-of-art results on Link Sign Prediction , which is the main machine learning task in signed networks [7]. To the best of our knowledge, none of the existing GNN methods has paid special attention to signed bipartite networks. It's the first time to introduce GNNs to signed bipartite networks. To summarize, the major contributions of this paper are as follows: • By defining the signed relationship of the same set of nodes (e.g., agreement/disagreement), we provide a new perspective for analyzing signed bipartite networks, which can measure the unbalanced structure of the signed bipartite networks from the same set of nodes. • Combining two perspectives, we introduce a new layer-by-layer SBGNN model. Via defining new message functions, aggregation functions, and update functions, SBGNNs aggregate information from neighbors in different node sets and output effective node representations. • We conduct Link Sign Prediction experiments on four realworld signed bipartite networks including product review, bill vote, and peer review. Experimental results demonstrate the effectiveness of our proposed model.

RELATED WORK 2.1 Signed Graph Modeling
Signed networks are such social networks having both positive and negative links [8]. Balance theory [3,17] is the fundamental theory in the signed network field [24]. For classical signed networks, signed triangles are the most common way to measure the balance of signed networks [37]. Distinct from homogeneous networks, there are two types of nodes in in bipartite networks. For signed bipartite networks, [5] conducts the comprehensive analysis on balance theory in signed bipartite networks, using the smallest cycle in signed bipartite networks (i.e., signed butterflies).
To mine signed networks, many algorithms have been developed for lots of tasks, such as community detection [2,41], node classification [38], node ranking [35], and spectral graph analysis [28]. The Link Sign Prediction is the main machine learning task for signed networks [36]. Modeling balance theory usually leads to better experimental results on Link Sign Prediction [20]. For example, [25] extracts features by counting signed triangles and achieve good performance in Link Sign Prediction . Even with recently signed network embedding methods [4,21,29,44], balance theory will be the important guideline for designing models. Specifically, SiNE [44] designs an objective function guided by balance theory to learn signed network embeddings and outperforms feature-based methods. For signed bipartite networks, how to incorporate balance theory is a research-worthy problem.

Graph Representation Learning
Graph Representation Learning (or Network Representation Learning) is to learn a mapping that embeds nodes, or entire (sub)graphs, as points in a low-dimensional vector space [14]. The nodes in the graph is represented as node embeddings, which reflect the structure of the origin graph. The common methods for graph representation learning include matrix factorization-based methods [30], random-walk based algorithms [12,33,40], and graph neural networks. Specifically, Node2vec [12] extends DeepWalk [33] by performing biased random walks to generate the corpus of node sequences; and it efficiently explores more diverse neighborhoods. For various complex networks (e.g., bipartite networks [10] and signed networks [50]), researchers have also proposed a variety of embedding methods by adapting the random walk methods.
Recently, Graph neural networks (GNNs) have received tremendous attention due to the power in learning effective representations for graphs [46,47]. Most GNNs can be summarized as a message-passing scheme where the node representations are updated by aggregating and transforming the information from the neighborhood [11]. GNNs have a partial intersection but use the deep learning methods instead of matrix factorization and random walk and can better describe the network structure [45]. A lot of GNN models show a better performance than the shadow lookup embeddings [13,23,43]. Most GNNs are designed for unsigned social networks whose links are only positive. For signed networks, some signed GNNs [6,19] are proposed to model the balance theory using convolution or attention mechanism. But they cannot handle the signed bipartite networks, because there are no links between nodes in the same sets. It is not trivial to transfer these models to signed bipartite networks.

BALANCE THEORY IN SIGNED BIPARTITE NETWORKS
For signed networks, balance theory is one of the most fundamentally studied social theories, which is originated in social psychology in the 1950s [17]. It discusses that due to the stress or psychological dissonance, people will strive to minimize the unbalanced state in their personal relationships, and hence they will change to balanced social settings. Specifically, triads with an even number of negative edges are defined as balanced. However, previous researches on balance theory are focused on unbipartite signed networks, measuring balance theory in signed bipartite networks is less studied.
In this section, we give two perspectives to analyze balance theory in signed bipartite networks.

Signed Bipartite Networks
Firstly, we describe our datasets used in this paper. The first dataset is from the e-commerce website Bonanza 1 , which is similar to eBay 2 or Taobao 3 . Users can purchase products from a seller and rate the seller with "Positive", "Neutral", or "Negative" scores. In this dataset, U represents the buyers, and V represents the sellers. The next two datasets (i.e., U.S. Senate and U.S. House) are from the 1st to 10th United States Congress vote records. It is collected from the Govtrack.us 4 . The senators or representatives U can vote "Yea" or "Nay" for bills V, which is the positive or negative links respectively. The above datasets are collected and used by [5] 5 . The last dataset is the peer review data from a top computer science conference 6 . Reviewers U can give "SA" (Strong Accept), "A" (Accept), "WA" (Weak Accept), "WR" (Weak Reject), "R" (Reject), and "SR" (Strong Reject) to papers V after reviewing. We regard "SA", "A", and "WA" as positive links and "SR", "R" and "WR" as negative links. It's worth mentioning that in most computer science conference peer reviews, there is usually a rebuttal phase when authors can point out errors in the reviews and help clarify reviewers' misunderstandings. It has proven to play a critical role in the final decision made by the meta-reviewers and the reviewers 7 . Besides, during the rebuttal phase, the reviewers can see the scores of other reviewers and make adjustments to their review comments and scores based on the author's response and other reviewers' comments. Therefore, the peer review dataset is divided into two parts: Preliminary Review and Final Review. We list the statistics of datasets in Table 1. From Table 1, we can find that in different scenarios, the negative ratio varies. In the scenario of product reviews, the ratio of negative links is relatively lower (i.e., 0.02). Buyers rarely give bad rates to sellers. In the scenario of bill vote, the proportion of negative links increases comparing to the scenario of product reviews (i.e., 0.460 and 0.447). In many bills, it is more difficult for legislators to reach consensus due to different political standpoints. In the scenario of peer reviews, the ratio of negative links is higher than the ratios of positive links (i.e., 0.603 > 0.397). In the top conferences of computer science, the acceptance rate needs to be controlled (e.g., about 20% in CIKM 8 ), so reviewers usually raise their standards for reviewing the paper, which will have a greater probability of giving negative reviews. Surprisingly, after the rebuttal phase, the proportion of negative links has slightly risen (i.e., from 0.597 to 0.603).

Signed Caterpillars and Signed Butterflies
The "butterfly" is the most basic motif that models cohesion in an unsigned bipartite network, which is the complete 2×2 biclique [34]. Base on the butterfly definition, [5] extends it to the signed butterfly by giving signs to the links in classical butterfly isomorphism. Except for signed butterfly definition, [5] denotes "signed caterpillars" as paths of length that are missing just one link to becoming a signed butterfly. They use signed butterflies to investigate balance theory in signed bipartite networks.
According to the definition of [5], we use the notation ⃝ ⃝ ⃝ ⃝ to denote a signed butterfly isomorphism class that represents the links between U and V (i.e., 1 → ⃝ 1 , 1 → ⃝ 2 , 2 → ⃝ 1 , 2 → ⃝ 2 ). Due to the symmetry of the structure, we can get 7 non-isomorphic signed butterflies classes. We show them in Perspective 1 in Figure 2 and divide them into two categories, balanced and unbalanced. For example, isomorphism class ++++ and ----denote the classes having all positive or all negative links, respectively. In the scenario of peer reviews, we can interpret isomorphism class ++++ as the situations where reviewer 1 and reviewer 2 both give "Accept" to paper 1 and paper 2 (i.e., 1 → + 1 , 1 → + 2 , 2 → + 1 , 2 → + 2 ). Similarly, we can interpret isomorphism class ----as reviewer 1 and reviewer 2 both reject paper 1 and paper 2 (i.e., 1 . Except isomorphism class ++++ and ----, isomorphism class ++--, +-+-, and ----are balanced since they have an even number of negative links. In fact, the definition of signed butterflies can be viewed as analyzing the signed bipartite network from Perspective 1.

Signed Triangles in Signed Bipartite Networks
For signed bipartite networks, the nodes of the same set are not connected. Therefore, we propose a new sign construction process by judging the sign of the link from U to V. After sign construction, we have signed links between nodes in the same set. It means that we have two new signed networks for U and V after sign construction.
Sign Construction Balanced Unbalanced As shown in Perspective 2 in Figure 2, when 1 and 2 have links with same sign on 1 (i.e., , 1 → + 1 , 2 → + 1 or 1 → − 1 , 2 → − 1 ), we construct a positive links between 1 and 2 (i.e., ++ ⇒ + and --⇒ +). When 1 and 2 have different link signs on 1 (i.e., , 1 → + 1 , 2 → − 1 ,), we construct a negative links between 1 and 2 (i.e., +-⇒ -). Since U is a set of people nodes (e.g., Buyer, Congress, and Reviewer), the positive and negative links can be regard as agreement and disagreements. For V, the positive link can be viewed as similarity and vice versa. After constructing the sign links between nodes of the same types, we can use the balance theory analysis in the classical signed networks. We can calculate the ratio of balanced triads (i.e., Triads with an even number of negative edges) in all triads [26]. The signed triangles +++ and +-are balanced as the principle that "the friend of my friend is my friend, the enemy of my enemy is my friend".

Balance Theory Analysis
In this subsection, we analyze the balance theory in different datasets from different perspectives. From Perspective 1, we follow [5] and calculate the percentage each isomorphism class takes up of the total signed butterfly count in each dataset as "%". Besides, we also calculate "%E" as the expectation of signed butterflies when randomly reassigning the positive and negative signs to the signed bipartite network. For example, "%E" for the isomorphism class For Perspective 2, we count the percentage of each signed triangles as "%" and the expectation of such signed triangles as "%E". Similarly, "%E" in U for +--is where |E | = |E − U | + |E + U |, and E + U and E − U are the positive and negative edges in U, respectively. Since U is the set of people nodes, which is easy to describe, we only list the results of U.
From Table 2, we can find that the large majority of signed butterflies in signed bipartite networks are more balanced than expectation based on the link sign ratio in the given networks (e.g., 0.988 > 0.924 in Bonanza). For Perspective 2, signed triangles in signed networks are also more balanced than expectation (e.g., 0.989 > 0.950 in Bonanza). Although the perspectives are different, the conclusions are similar. In the scenario of peer reviews, after rebuttal phase, the balance of signed bipartite networks increased (i.e., 0.724 → 0.774 ↑ and 0.778 → 0.792 ↑). It shows that through authors' feedback and reviewers' discussions, the reviewers' opinions have become more balanced, although ratio of the negative links increased. From Perspective 1, the ratio of isomorphism class ++++, +-+-, ----increased (i.e., 0.109 → 0.115 ↑, 0.156 → 0.215 ↑, and 0.239 → 0.315 ↑), which means that reviewers made a more balanced adjustment to their review comments. For Perspective 2, the ratio of signed triangles +++ increased from 0.327 to 0.446 ↑, which reflects that the reviewers are more balanced and consistent after the rebuttal phase.

PROBLEM FORMULATION
In this section, we give the definition of Link Sign Prediction , which can be regarded as the main machine learning task for signed bipartite networks.
Consider a signed bipartite network, .., |V | } represent two sets of nodes with the number of nodes |U| and |V |. E ⊂ U × V is the edges between U and V. E = E + E − is the set of edges between the two sets of nodes U and V where E + E − = ∅, E + and E − represent the sets of positive and negative edges, respectively. Given G = (U, V, E), and from two different sets (their link sign is not observed), the goal is to find a mapping function ( , ) → {−1, 1}. For network embeddings methods or GNNs, it will learn the representation of the node and to get embeddings ∈ R and ∈ R , and use the embeddings to get the results by ( , ) → {−1, 1}.  Vanilla GNNs usually follow a message passing scheme where the node representations are updated by aggregating and transforming the information from the neighborhood [49].
Specifically, for a graph G = (V, E), where V = { 1 , 2 , ... | | } is the node set and E ⊂ V × V is the edge set. The goal of GNNs is to learn node representation ℎ for node based on an iterative aggregation of local neighborhoods.
For the -th layer of a GNN, it can be written as: where Msg, Agg, Upt are functions for message construction, message aggregation, and vertex update function at the -th layer [27].
Most Graph Neural Networks [23] are designed for unsigned classical social networks. They design the Msg and Agg function such as mean [23], max [13], sum [48] or attention [43]. In this paper, we follow the message passing scheme and show the SBGNN Layer in Figure 3 including the design of Msg, Agg and Upt.

Message and Aggregation Function
As we discussed in Section 3, comparing to traditional unsigned networks, the links in signed bipartite networks is cross-set and complex (e.g., → +/− ).
The message function of vanilla GNNs cannot be directly applied to signed bipartite networks. As shown in Figure 3, we design a new message function to aggregate messages from different sets of neighborhoods . We define that Set 1 refers to the set of node with different types, and Set 2 refers to the set of nodes with the same types. Message from Set 1 and Set 2 can be viewed as the modeling of Perspective 1 and Perspective 2 in Section 3, respectively.

Message from Set 1 .
For Set 1 , the type of the set is different from the type of the current node, and because it is a signed network, its links include both positive and negative relationships. Neighborhood nodes under positive and negative links have different semantic relations. So we use → + and → − to aggregate the message from to with positive and negative links and → + and → − to aggregate the message from to with positive and negative links. For example, in Figure 3, for the purple circle 1 , the red square 2 and the green square 4 are the positive and negative neighborhoods, respectively.
5.1.2 Message from Set 2 . As we said before, Set 2 is the node set of the same type. However, there are no links between nodes of the same type, so we need to construct an sign link between nodes of the same type through sign construction in Section 3. After sign construction, we can aggregate message for the positive and negative links from to with → + and → − , respectively and to with → + and → − , respectively.
In Figure 3, for the purple circle 1 , the green circle 3 and the red circle 5 are the positive and negative neighborhoods because of the link 1 → − 2 → − 3 and 1 → − 2 → + 5 . For the purple square 5 , 1 and 2 are the negative neighborhoods, 3 and 4 are the positive neighborhoods based on our sign construction.
In summary, we can definite the message from Set 2 as follows: where N → − ( ), N → − ( ), N → + ( ), and N → − ( ) are positive and negative neighborhoods for and , respectively.

Aggregation Design.
For message aggregation, it is commonly a differentiable, permutation invariant set function (e.g., mean, max, and sum) that take a countable message set { ( )| ∈ N ( )} as input; and output a message vector . In this paper, we use mean (Mean) and graph attention (Gat) aggregation functions.
For the Mean aggregation function, we get ( ) and ( ) by Mean the message of neighborhoods: where is a relationship of links to (e.g., , → + ) and is a relationship of links to (e.g., , → + ).
For a graph attention function, it will firstly compute for node and node by the attention mechanism ì a and LeakyReLU nonlinearity activation function (with negative input slope = 0.2) as: where ∥ is is the concatenation operation, · ⊤ represents transposition, N ( ) is the neighborhoods of node under the definition of (e.g., , → + ) , W is the weight matrix parameter. Then we can compute ( ) and ( ) with : where is a relationship of links to (e.g., , → + ) and is a relationship of links to (e.g., , → + ). The attention aggregation can be seen as a learnable weighted average function.

Update Function
For our vertex update function, we concatenate the messages from different neighborhoods with origin node features and apply it to an Mlp to get the final node representation: where ∥ is the concatenation operation. More specificlly, the Mlpis a two-layer neural networks with Dropout layer and Act activation function: where 1 , 1 and 2 , 2 is the parameters for this MLP, and Dropout is the dropout function ( = 0.5 in this paper) and Act is the activation function (e.g., PReLU [15] in this paper) .

Loss Function
After getting embeddings ∈ R and ∈ R of the node and , we can use following methods to get the prediction value for → . The first one is the product operation: where · ⊤ is the transpose function and sigmoid is the sigmoid function ( ) = 1 1+ − . This method should keep the embedding dimension the same (i.e., = ). Another method is to use an Mlp to predict the values by where Mlp is a two layer neural networks, ∥ is the concatenation operation. The Mlp can be viewd as the Edge Learner in [1]. After getting the prediction values, we use binary cross entropy as the loss function: where is the rescaling weight for the unblanced negative ratios (It is the weights inversely proportional to class frequencies in the input data); is the ground truth with mapping {−1, 1} to {0, 1}.

Training SBGNN
With the design of our SBGNN model, the training procedure is summarized in Algorithm 1. Back propagation, update parameters. 12: end for 13: return U , V Algorithm 1 demonstrates that our SBGNN is a layer-by-layer architecture design, where can be any powerful GNN aggregators like Mean or Gat.

EXPERIMENTS
In this section, we evaluate the performance of our proposed SBGNN on real-world datasets. We first introduce the datasets, baselines, and metrics for experiments, then present the experimental results of SBGNN and baselines. Finally, we analyze our models from parameter analysis and ablation study.
6.1 Experimental Settings 6.1.1 Datasets. As previously discussed in Section 3.1, we choose four datasets for this study, namely, Bonanza, U.S. House, U.S. Senate, and Review (we use Final Review as the Review dataset). Following the experimental settings in [5], we randomly select 10% of the links as test set, utilize a random 5% for validation set, and the remaining 85% as training set for each of our datasets. We run with different train-val-test splits for 5 times to get the average scores.
6.1.2 Competitors. We compare our method SBGNN with several baselines including Random Embeddings, Unsigned Network Embeddings, Signed/Bipartite Network Embeddings, and Signed Butterfly Based Methods as follows.
Random Embeddings: It generates dimensional random values from a uniform distribution over [0, 1) (i.e., = ( 1 , 2 , ..., ), ∈ [0, 1)). Given embeddings and , we concatenate them and use a Logistic Regression (Lr) to predict the value of and . Lr will be trained on the training set, and make predictions on the test set. Since Lr has learnable parameters, this method can be viewed as the lower bound of the graph representation learning methods.
Unsigned Network Embeddings: Comparing random embeddings, we use some classical unsigned network embedding methods (e.g., DeepWalk [33] 9 , Node2vec [12] 10 , LINE [40] 11 ). By keeping only positive links, we input unsigned networks to such unsigned network embedding methods to get embeddings for and . As same as Random Embeddings, we concatenate embeddings and , and use Lr to predict the sign of links.
Signed/Bipartite Network Embedding: We use Signed or/and Bipartite Network Embedding as our baselines. More specifically, we use SiNE [44] 12 to learn the embeddings for and after sign link construction in Section 3. For BiNE [10] 13 , we remove the negative links between U and V . We use BiNE to get embeddings and and use Lr to predict the sign of links with concatenating and . Compared with the unsigned network embeddings methods, we try to let the representation learn the structural information (e.g., links between U and V and the link sign) instead of just relying on the downstream classifier. SBiNE [51] is a representation learning method for signed bipartite networks, which preserves the first-order and second-order proximity. Instead of a two-stage model, SBiNE uses single neural networks with sigmoid nonlinearity function to predict the value of and .
Signed Butterfly Based Methods: Based on the analysis of signed butterfly isomorphism, [5] proposes a variety of methods for Link Sign Prediction , including SCsc, MFwBT, and SBRW 14 . Specifically, SCsc is a balance theory guided feature extraction method. MFwBT is the matrix factorization model with balance theory. SBRW is the signed bipartitle random walk method. Due to the findings about receiving aid in prediction from balance theory will always perform better than the methods that only use generic signed network information [5], we take the SCsc as the most competitive baseline for our SBGNN model.
Signed Bipartite Graph Neural Networks: For our SBGNN, we try two different aggregation design (i.e., Mean and Gat) and remark it as SBGNN-Mean and SBGNN-Gat, respectively.
For a fair comparison, we set all the node embedding dimension to 32 which is as same as that in SBiNE [51] for all embedding based methods. For other parameters in baselines, we follow the recommended settings in their original papers. For embedding methods, we use the balanced class weighted Logistic Regression in Scikit-learn [32] 15 . For SBiNE, we use PyTorch [31] to implement it by ourselves. For our SBGNN, we also use PyTorch to implement our model. We use Adam optimizer with an initial learning rate of 0.005 and a weight decay of 1e-5. We run 2000 epochs for SBGNN and choose the model that performs the best AUC metrics on the validation set.

Evaluation Metrics.
Since Link Sign Prediction is a binary classification problem, we use AUC, Binary-F1, Macro-F1, and Micro-F1 to evaluate the results. These metrics are widely used in Link Sign Prediction [4,20]. Note that, among all these four evaluation metrics, the greater the value is, indicating the better the performance of the corresponding method.

Experiment Results
We show the results in Table 3. We have bolded the highest value of each row and underlined the second value. From Table 3, we make the following observations: • Even with random embedding, Lr can still achieve a certain effect on Link Sign Prediction (i.e., AUC > 0.5). It demonstrates that the downstream classifier has a certain predictive ability for Link Sign Prediction when it is regarded as a two-stage model. • After using network embeddings, the graph structure data is modeled into the node representation, which improves the prediction results. Even the worst-performing LINE outperforms random embedding by 17.3%, 13.6%, 12.3%, and 9.4% on AUC in Bonanza, Review, U.S. House, and U.S. Senate, respectively. It demonstrate that graph structure information is helpful for Link Sign Prediction . In unsigned network embedding methods, Node2vec has made the best results in the unsigned network embeddings methods. We guess that the biased random walk mechanism may be able to explore the graph structure more effectively. • For signed or bipartite model (i.e., SiNE and BiNE), the information for sign and bipartite network structure can contribute to the node representation learning. SiNE is more effective than BiNE (e.g., AUC in Bonanza (0.6088 > 0.6026)), indicating that link sign information may be more important than the link relationship.
14 https://github.com/tylersnetwork/signed_bipartite_network 15 https://scikit-learn.org/stable/index.html The performance of SBiNE is not as good as that in [51]. This can be due to the fact of different data splits and implementation details.

Parameter Analysis and Ablation Study
In this subsection, we conduct parameter analysis and ablation study for our SBGNN model. We choose the U.S. House as our dataset and select 85% training edges, 5% validation edges, and 15% test edges as before.
6.3.1 Parameter Analysis. We analyze the number of our SBGNN Layer and the dimension of embeddings. For the number of our SBGNN Layer , we vary from {0, 1, 2, 3, 4}. Note that, = 0 means there is no GNN Layer is used (i.e., just lookup embeddings is used), so the results for SBGNN-Mean and SBGNN-Gat are same. For Figure 4(a), we can find that GNN Layer is more effective than the lookup embedding methods. For SBGNN-Mean, the best is 1, which AUC is 0.8481. But for SBGNN-Gat, two SBGNN Layers will get a better results. For the dimension of SBGNN, we choose the value from {4, 8, 16, 32, 64, 128} to analyze the effects of dimensions. From Figure 4(b), we can find that as the value increases from 4 to 32, AUC on SBGNN-Gat increases from 0.8056 to 0.8485; AUC on SBGNN-Mean increases from 0.8053 to 0.8447.  After 32 for SBGNN-Mean and 64 for SBGNN-Gat, the AUC value slightly descrease. This result can be due to the reason that large dimension will cause the difficulties of training embeddings.

Ablation Study.
For the ablation study, we investigate the effect of different aggregation and prediction functions. Firstly, as we discussed in Section 6.2, Gat aggregator is better than Mean aggregators. We further investigate the effect for our message function design. From Table 4, we can see that without message from Set 1 (i.e., w/o Set 1 ), SBGNN-Gat and SBGNN-Mean descrease 0.9% and 0.3%, respectively; without Set 2 message (i.e., w/o Set 2 ), SBGNN-Gat and SBGNN-Mean descrease 0.5% and 1.8%, respectively. It demonstrates that both message from Set 1 and Set 1 is useful for the SBGNN model. As we discussed in Section 5.3, the prediction can be achieved by product operation or Mlp. We replace it with a simple Lr layer or a two-layer Mlp. From Table 4, we can find that Mlp is much better than simple Lr but not better than product operation.

CONCLUSIONS AND FUTURE WORK
In this paper, we focus on modeling signed bipartite networks. We first discuss two different perspectives to model the signed bipartite networks. Through sign construction, the new perspective can count the signed triangles in the same node type networks. It obtains consistent results with signed butterfly analysis. We further use these two perspectives to model peer review and find that after rebuttal, the balance of reviewers' opinions improved. It shows that the rebuttal mechanism makes the reviewer's opinions more consistent. Under the definition of a new perspective, we propose a new graph neural network model SBGNN to learn the node representation of signed bipartite graphs. On four real-world datasets, our method has achieved state-of-the-art results. Finally, we conducted parameter analysis and ablation study on SBGNN.
In future work, we will explore signed bipartite networks with node features, which can improve the Link Sign Prediction [22] and node classification [38]. For example, in the prediction of bill vote, if the node features can be modeled, such as the political standpoint of the congress, it will be more effective in predicting the vote results. Besides, we will also try to introduce signed bipartite graph neural networks into recommender system [39].