{'fraction_non_alphanumeric': 0.03534088433892398, 'fraction_numerical': 0.007895883249836637, 'mean_word_length': 4.365176745544844, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A Framework of Transferring Structures Across Large-scale Information Networks

The existing domain-specific methods for mining information networks in machine learning aims to represent the nodes of an information network into a vector format. However, the real-world large-scale information network cannot make well network representations by one network. When the information of the network structure transferred from one network to another network, the performance of network representation might decrease sharply. To achieve these ends, we propose a novel framework to transfer useful information across relational large-scale information networks (FTLSIN). The framework consists of a 2-layer random walks to measure the relations between two networks and predict links across them. Experiments on real-world datasets demonstrate the effectiveness of the proposed model.


I. INTRODUCTION
I NFORMATION networks [1] is a kind of structure-based data, e.g., academic citation networks, which employ network topologies to save a part of information directly. Largescale information network ranges the size from hundreds of nodes to millions and billions of nodes [2]. The large volume of nodes make complex connections over the network and contains complex data structures than normal information networks. To fully analyze such kind of information networks is a quite challenging problem especially in machine learning.
Network representation, also known as network embedding, allows analyzing the network structure and mining the information behind the structure in a machine learning perspective [3], [4]. By generating a latent representation space in relatively low dimensions from the interactions in high dimensions, network representation inputs a structured data of graph [5] and outputs the embeddings of the graph in a specific dimensional space. It guarantees the correspondence between community structure in the input graph and its embeddings. The main advantage of network representation is that the learned representations encode community structure, so it can be easily exploited by simple and standard classifiers [6]. For the purpose of using network analysis on structured data, a series of models are proposed based on DeepWalk [7], which trains a natural language model on the random walks generated by the network structure. Denote a random walk w vs that starts from a root node v s , DeepWalk slides a window in a length of 2w + 1, and maps the central node v i to its representation f (v i ). Hierarchical Softmax factors out the probability distributions P r(v i±d |f (v i )), where d = {1, · · · , w}, corresponding to the paths staring at v i and going over all other nodes in the random walk. The representation f is updated to maximize the probability of v i co-occurring with its context {v i±d , d = {1, · · · , w}}. Random walk based DeepWalk shows promising results on large-scale network representation if the datasets have a satisfying structure.
LINE and Node2Vec are the other two structure-based network representation models that improves the performance of DeepWalk. LINE [2] preserves both local and global network structure by first-order proximity and second-order proximity respectively and suitable for all kinds of networks, i.e., directed and undirected networks and weighted and unweighted networks. Node2Vec [8] explores the diverse neighborhoods of nodes in a biased random walk procedure with search bias α.
Above mentioned models are inspired from recent advancements in unsupervised feature learning and the language modeling from sequences of words to vectors or networks. They contributed to the network analysis by modeling a stream of short random walks. Different from traditional representation learning, the latent feature learning of network representation captures neighborhood similarity and community membership in topologies. The potential of these models in real-world scenarios is their good performance in large heterogeneous networks.
When one network lacks of connections, the existing network representation methods cannot perform well in domainspecific network representations. Meanwhile, the performance of the domain-specific network representations methods decreases sharply when transferred from one network to another relational network. If we develop a framework that successfully combines the advantages of network representation and domain adaptation, structure transfer will extremely benefit to real-world large-scale information network representations.
In this paper, we consider the following challenges of developing the framework for transferring the network structures across large-scale information networks.
• Challenge 1: How to effectively predict links between walk mapping nodes across relational networks for the purpose of improving performance of network representation in the target network? • Challenge 2: How to transfer the random walks in the source network to the target network based on the similarity measurement achieved in Challenge 1? To this end, we propose a framework of transferring structures across large-scale information networks (FTLSIN) which implements an unsupervised feature learning for scalable networks. Our framework is built by a 2-layer random walks to generate a neighborhood of nodes in the target network with a secondary from the learned walks in the source network which measures the similarity and predicts links across networks. Experimental results on real-world datasets empirically demonstrate that our framework achieves better performances compared to the state-of-the-art network representations.

II. PROBLEM STATEMENT
The problem of transferring structures across large-scale information networks is formulated as follows. Suppose we have a source domain D s and a target domain D t , where the source domain has a source networks G s = (V s , E s ) with its corresponding label space Y s , and the target domain has a target network G t = (V t , E t ) with its label space Y t . Both networks are unweighted. For a cross-domain classification problem < T t , T s , (x t test , y t test ) >, we firstly implement a latent feature learning procedure from topology structures of G s and G t as a maximum likelihood optimization problem and then learn the labels Y t in the target domain with standard classifiers as an evaluations of the cross-domain network representations.
In this paper, let f : V → R d be the mapping function from nodes to feature representation, where d refers to the lower-dimensions of our representation, f s are specially designed for the source network and the target network respectively, i.e., f s : V s → R d and f t : V t → R d . As a sampling strategy, we define a neighborhood of nodes N S (u) ⊂ V for every node in the source network and in the target network, where N S (u s ) ⊂ V s , u s ∈ V s , N S (u t ) ⊂ V t and u t ∈ V t . By predicting the latent feature space R d , our proposed framework can be applied to any (un)directed and (un)weighted network across domains.

A. Skip-gram in FTLSIN
FTLSIN shown in Fig. 1 learns random walks by Skipgram and outputs the network representations from the input of networks in source domain and target domain, respectively. Skip-gram [9] is a language model exploiting word orders in a sequence and assuming that words closer are statistically more dependent or related. We employ the Skip-gram architecture to FTLSIN which treats the nodes in a sequence and fully uses the structures to make network analysis.
Given a current node u s in the source network within a certain window, we have a FTLSIN Skip-gram for source networks by maximizing the following log-likelihood function of f s in observing a neighborhood of N S (u s ): Given a node u t in the target network with a certain window, we have a FTLSIN Skip-gram for the target network by maximizing the following log-likelihood function of f t in observing a neighborhood of N S (u t ): Following the standard assumptions of Node2Vec model [8], conditional independence and symmetry in feature space are defined in Eqs. (3) and (4), respectively.
In our proposed FTLSIN Skip-gram (see in Algorithm 2), the network neighborhood strategy applied on the target network can be different from the ones on source networks. Meanwhile, the window length and optimization function f s and f t set in FTLSIN Skip-gram also can differ from networks.

B. 2-layer Random Walks in FTLSIN
The FTLSIN 2-layer random walks consists of a bottomlayer random walk and a top-layer random walk. In Fig. 1, the 2-layer random walks measures the likelihood between super-nodes {v , x } based on its learning of random walk w s i . The top-layer maps two nodes {v, x} in the target network to the corresponding super-nodes {v , x } in a source network within a node mapping procedure and a walk mapping procedure. The Algorithm of the 2-layer random walks in FTLSIN is as shown in Algorithm 1.
Given a random walk of node in either a source network u s or a target network u t , u is in a fixed length of l, i.e., the length of u s is l s and the length of u t is l t . 2-layer random walks allows l s different from l t .
Let v i denote the ith node in the walk, where the start node is v 0 = u and all the nodes in the walk follows the distribution: f node ← Node mapping on w s i by Eq. (11).

5:
w v x ← Wwalk mapping on w s i and f node by Eqs. (12)- (13), where v , x ∈ V s . 6: end for 7: W t ← Apply bottom-layer random walk to process the target network, Eqs.
where π v s x s and π v t x t are the unnormalized transition probability between nodes v s and x s , and between nodes v t and x t ; and Z is the normalizing constant.
1) Bottom-layer Random Walk: The design of the bottomlayer random walk is for the network representation both in the target network and in the source network. We employ parameters p and q to guide the walk by considering the network neighborhood. In order to determine which node in the neighborhood have a higher probability to be connected into the random walk, the search bias α is employed into Eqs. (7) and (9): where w v s x s is the weight on edge (v s , x s ).
where d t s x s is the shortest path between nodes t s and x s through node v s .
where w v t x t is the weight on edge (v t , x t ).
where d t t x t is the shortest path between nodes t t and x t through node v t . f t ← Apply Eq. (2)-(4) to optimize f t . 4: end for 5: return f t 2) Top-layer Random Walk: For the random walk in the top-layer, we define a node mapping procedure and a walk mapping procedure. The node mapping procedure starts from one node v ∈ V t in the target network to a node set v ∈ V s i in the source network. The walk mapping procedure learns from a walk w s ∈ {W s i } in the source network to a new walk w t ∈ W t in the target network.
Following the assumption of transfer learning [10], the scale of networks in the source domain is much larger than the scale of the network in the target domain, i.e., |V s i | |V t | or |E s i | |E t |. The node mapping procedure links a node in target network and a set of nodes in source network. Thus, we employs the definition of super-graph and supernode to process the node mapping procedure. Specifically, the node set v is denoted as a super-node.
A super-graph [11] is represented as G = (V, E, G, F), where V is a finite set of graph-structure nodes. E ⊂ V × V denotes a finite set of edges, and F : E → G is an injective function from E to G, where G is the set of single-attribute graphs. A node in the super-graph, represented by a singleattribute graph, is called a super-node.
As above, our node mapping procedure measures the likelihood of a node in the target network and a super-node in the source network, i.e., f node : v t → V s i .
In the walk mapping procedure, walk set of target network W t is jointly determined by the node mapping function f node and weighted random walk kernel [11] on super-graph in source network. The walks over the source network W s = {w s i } is naturally within a super-graph. The edges forming a w s i links two super-nodes, as shown in Fig. 1 (d). Within the top-layer random walk, an edge weight in a target walk (w v x in w t i ) is formed by two terms in Eq. (12). The former term is contributed by the virtual weight in the target network w v t x t , and the latter term is contributed by the learning weight from a walk mapping w t vv xx .
where P (x s |v s ) = 1/d v s x s .

A. Datasets
We select two academic citation networks as the datasets. Both of them are for the multi-class classification problem. Nodes are denoted as papers in these networks. DBLP dataset [12] (source network), consisting of bibliography data in computer science has been used widely in network and graph analysis [13], [14]. Each paper may cite or be cited by other papers, which naturally form a citation network. The network in this dataset abstracts a list of conferences from 4 research areas, i.e., database, data mining, artificial intelligence and computer vision.   CiteSeer-M10 dataset [12] (target network) is a subset of CiteSeerX data which consists of scientific publications from 10 distinct research areas, i.e., agriculture, archaeology, biology, computer science, financial economics, industrial engineering, material science, petroleum chemistry, physics and social science.

B. Setups
Our experiment evaluates the latent feature representations on standard supervised learning task: linear SVM classification. We choose the linear classifier instead of non-linear classifier or sophisticated relational classifiers in order to reduce the impact of complicated learning approaches on the classification performance. For evaluations, we randomly partition the dataset in the target domain into two nonoverlapping sets for training and testing by 9 groups of training percents, {0.1, 0.2, · · · , 0.9}. We repeat the above steps for 10 times and thus receive 10 copies of training data and testing data. The reported experimental results are the average of the ten runs and their variance. Fig. 3 show power law distributions [15] on the experiment datasets and their random walks, which obey the assumptions of the random walk that if the degree distribution of a connected graph follows a power law distribution, the frequency which the nodes appear in the short random walks will also follow a power law distribution [7].

Fig. 2 and
We implement the following random walk based domainspecific network representation models for comparison. The benchmark models are applied with the FTLSIN Skip-gram for source networks in Eqs. (1), (3)-(4). 1) DeepWalk [7]: This approach learns d-dimensional feature representations by simulating uniform random walks. The sampling strategy in DeepWalk can be seen as a special case of FTLSIN with bottom-layer random walk in p = 1 and q = 1.

2) LINE [2]
: This approach learns d-dimensional feature representations in two separate phases. In the first phase, it learns d/2 dimensions by BFS-style simulations over immediate neighbors of nodes. In the second phase, it learns the next d/2 dimensions by sampling nodes strictly at a 2hop distance from the source nodes.
3) Node2Vec [8]. This approach learns d-dimensional feature representations by BFS-style simulations over immediate neighbors of nodes. The sampling strategy in Node2Vec is also a special case of FTLSIN with bottom-layer random walk in p = 1 and q = 1.

D. Parameters Setting
The parameter settings used for FTLSIN are in line with typical values used for DeepWalk, LINE and Node2Vec. Specially for source and target networks, we set the dimensions of feature representation as d = 128, set the walk length as l = 80, set the number of walks of every source node as k = 10, and set the window size as r = 10. In this way, the total number of walks over a input network is w = SampleSize × k, and the shape of walk sets are in w × l. The parameters of search bias α is set in p = 1, q = 1.

E. Experimental Results
The node feature learning by network representations are input to a one-against-all linear SVM classifier [16]. We use Macro-F1 and Micro-F1 for comparing performance and the results are shown in Table II. These two measures are popular just like the classification accuracy performance in data mining areas [17], [18], [19].
Representation Analysis. Fig. 4 (a) illustrate the feature spaces of dblp by FTLSIN bottom layer random walk, Fig.  4 (b) illustrate the feature spaces of dblp by FTLSIN 2-layer random walks. These two illustrations show almost the same distributions in feature spaces and get good mappings in a low dimension than PCA, LLE and Laplacian based network representaitons. Effectiveness of search priority in random walks. In Table II, DeepWalk and LINE show the worse performance than our FTLSIN and Node2Vec, which can be explained by its inability to reuse samples, a feat that can be easily done using the random walk. The outstanding of Node2Vec among benchmark models indicates the exploration strategy is much better than the uniform random walks learned by DeepWalk and LINE. Meanwhile, the poor performance of DeepWalk and LINE is mainly because the network structure is rather sparse, with noises and only contains limited information. FTLSIN and Node2Vec are both good performing on M10 network with above advantages, as parameter of search bias α adds the flexibility in exploring local neighborhoods prior to global network.
Importance of information from source domain. Table II shows that FTLSIN outperforms the domain-specific benchmark models, which uses topological information from the source domain to learn the network representation in the target domain. When we add a top-layer in 2-layer random walks, the information in the source network are transferred to the source network by adjusting the weights on the edges of the target network.

V. CONCLUSION AND FUTURE WORK
In this paper, we propose a solution for a new scenario in network representation, that is transferring structures across networks with 2-layer random walks. Our framework effectively improves the performance of latent feature learning in large-scale citation networks. Meanwhile, it reduces learning difficulties of data sparsity and noises. Future works include FTLSIN with multiple labels and deep network representation [20], [21].