{'fraction_non_alphanumeric': 0.04567734820170333, 'fraction_numerical': 0.019729183260829605, 'mean_word_length': 4.385744926579772, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 3, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Community Detection in General Hypergraph via Graph Embedding

Conventional network data has largely focused on pairwise interactions between two entities, yet multi-way interactions among multiple entities have been frequently observed in real-life hypergraph networks. In this article, we propose a novel method for detecting community structure in general hypergraph networks, uniform or non-uniform. The proposed method introduces a null vertex to augment a non-uniform hypergraph into a uniform multi-hypergraph, and then embeds the multi-hypergraph in a low-dimensional vector space such that vertices within the same community are close to each other. The resultant optimization task can be efficiently tackled by an alternative updating scheme. The asymptotic consistencies of the proposed method are established in terms of both community detection and hypergraph estimation, which are also supported by numerical experiments on some synthetic and real-life hypergraph networks.


Introduction
Community detection has attracted increasing attention from both academia and industry in the past few decades, and has a wide spectrum of applications in domains, ranging from social science (Ji et al., 2016;Newman et al., 2002;Zhao et al., 2011;Lee et al., 2017), life science (Chen and Yuan, 2006;Nepusz et al., 2012) to computer science (Tron and Agarwal et al., 2005). The community structure has been widely observed in many real-life networks, which usually means that entities within the same community tend to interact much more often than across communities. In literature, most conventional network analysis focus on pairwise interactions between two vertices (Zhao et al., 2012;Lei et al., 2015;Loyal and Chen, 2020;Sengupta and Chen, 2018). However, the complexity of real-life networks is usually beyond pairwise interaction, and multi-way interactions among vertices arise naturally. For example, in an academic collaboration network, it is often the case that several researches work together on a research project; in a protein-to-protein interaction network, a metabolic reaction usually involves multiple proteins. Under such circumstances, a hypergraph network provides a more faithful representation and retains richer information than a merely vanilla graph network, such as each research project being represented by a hyperedge consisting of multiple vertices being researchers, or each metabolic reaction being a hyperedge consisting of multiple vertices being proteins. In this paper, we are interested in detecting community structure in a hypergraph network, where vertices within the same community share more similar connection patterns compared with vertices in different communities.
To detect communities in a hypergraph network, most existing methods convert the hypergraph into a weighted graph, and then existing graph community detection methods can be applied. For instances, Kumar et al. (2021) defined the hypergraph modularity as the modularity of the weighted graph and applied standard modularity maximization algorithms for community detection, Lee et al. (2021) extended the graph-likelihood-based convex relaxation methods (Li et al., 2020b) on the adjacency matrix of the weighted graph, and Ghoshdastidar and Dukkipati (2017a) conducts spectral clustering on the weighted graph Laplacian for hypergraph community detection. As pointed out in Ke et al. (2021), such conversion will suffer from information loss and lead to suboptimal community detection performance. To circumvent this disadvantage, spiked tensor model (Kim et al., 2017) and Tensor-SCORE method (Ke et al., 2021) were proposed to conduct community detection on the hypergraph adjacency tensor directly.
Note that most existing hypergraph community detection methods focus on uniform hypergraph only Dukkipati, 2014, 2015b;Lee et al., 2021), where all the hyperedges consist of exactly the same number of vertices. When it comes to non-uniform hypergraph, they have to decompose a non-uniform hypergraph into a collection of uniform hypergraphs with different orders, due to the difficulty of lacking an appropriate adjacency tensor for non-uniform hypergraph. It was only until recently that Ouvrard et al. (2021) proposed a heuristic method to construct an adjacency tensor for general hypergraph by adding m − m 0 null vertices, where m and m 0 are the maximum and minimum cardinality of a hyperedge in the hypergraph, respectively.
In this paper, we propose a novel method for detecting community structure in general hypergraph networks, where a general hypergraph can be uniform or non-uniform. The proposed method consists of an augmentation step and an embedding step. The augmentation step adds a null vertex to the hyperedges with smaller cardinality, and converts the hypergraph into a uniform multi-hypergraph Zhang, 2014, 2015), which allows vertices to appear multiple times in one hyperedge. The embedding step is formulated in a regularization form with tensor decomposition, which represents each vertex as a lowdimensional numerical vector and encourages vertices within the same community to be close in the embedding space. The proposed hypergraph embedding model is flexible and general, which includes the hypergraph stochastic block model (hSBM), also known as hypergraph planted partition model (Ghoshdastidar and Dukkipati, 2017a), as its special case. It also accommodates heterogeneity among vertices by allowing vertices within a community to fluctuate in all directions. This is in sharp contrast to hSBM, which assumes vertices within a community share the same spectral embedding vector. It is also more general than the hypergraph degree-corrected block model (hDCBM; Ke et al. 2021;Yuan et al. 2021), un-der which spectral embeddings of vertices within a community are along the same direction and only differ by their magnitudes. The advantage of the proposed method is supported in both asymptotic theories and numerical experiments on a number of synthetic and real hypergraph networks.
The main contribution of this paper is three-fold. First, we propose a novel hypergraph embedding model (HEM), which consists of an augmentation step and an embedding step.
The augmentation step introduces only one null vertex rather than multiple ones as suggested in Ouvrard et al. (2021), which is more efficient and greatly facilitates the subsequent community detection. The embedding step, to the best of our knowledge, is the first statistical framework that extends the latent space model for graph network to general hypergraph network with theoretical guarantees. Second, a joint modeling framework is developed for simultaneously conducting hypergraph estimation and community detection. Third, we establish the asymptotic consistencies of the proposed method in terms of both hypergraph estimation and community detection in sparse hypergraph network. Particularly, the consistencies hold as long as the link probability is of the order s n n 1−m log n in an mth-order hypergraph with n vertices. This result compares favorably with the existing sparsity results in literature (Ke et al., 2021;Ghoshdastidar and Dukkipati, 2017a,b), not to mention that the proposed method also achieves fast convergence rate in hypergraph estimation.
The rest of the paper is organized as follows. Section 2 provides a quick review of some preliminaries on hypergraph and tensor. Section 3 presents the details of the proposed hypergraph augmentation and embedding formulations, and an efficient optimization algorithm.
Section 4 establishes the asymptotic consistencies of the proposed method. Section 5 examines the numerical performance of the proposed method on both synthetic and real-life hypergraph networks. Section 6 concludes the paper, and technical proofs and necessary lemmas are contained in the supplementary files.

Preliminaries
To begin with, we introduce some basic concepts of hypergraph and tensor that will be used extensively in the sequel. A general hypergraph is denoted as For definition of directed hypergraph, interested readers may refer to Gallo et al. (1993) and Ouvrard et al. (2021). A tensor A = (a i 1 ...im ) ∈ R I 1 ×...×Im is a cubical tensor if I 1 = ... = I m = n, for some n ∈ Z + . More formally, such a tensor is called an mth-order tensor of dimension n. A cubical tensor A is super symmetric if a i 1 ...im = a π(i 1 )...π(im) , for any possible permutations π ∈ S m , the symmetric group of degree m. A mth-order super symmetric tensor I ∈ {0, 1} n×...×n is called an identity tensor if I i 1 ...im = 1 when i 1 = ... = i m and 0 otherwise.
Let M (j) ∈ R Q j ×I j be m matrices, then the mode j product (Kolda and Bader, 2009) between A and M (j) is defined as Recall that a multi-set {{i 1 , i 2 , ..., i m }} is an extension of set in that some elements may appear multiple times. Herein, to distinguish from a set, we use double curly braces {{·}} to denote a multi-set as in Bahmanian and Sajna (2015) and Kovačević and Tan (2018). Clearly,

Community detection in hypergraph
This section proposes a novel community detection method for general hypergraphs. The key idea is to introduce a null vertex so that hyperedges with smaller cardinality can be augmented, and the non-uniform hypergraph is transformed into a special uniform multihypergraph, where the null vertex may appear multiple times in some hyperedges. Then the proposed method proceeds to seek for a suitable numeric embedding that embeds all vertices of the hypergraph into a low dimensional Euclidean space, where vertices belong to the same community tend to have shorter distance.

Hypergraph augmentation
Let H(V, E) be a hypergraph, possibly non-uniform, with V = [n] and range m ≥ 2, which is the maximum cardinality of a hyperedge in E. One of the key challenges of analyzing nonuniform hypergraph is due to the unequal hyperedge cardinalities, leading to the ambiguity of discriminating hyperedges and their proper subsets.
To circumvent this difficulty, we introduce a null vertex, denoted as v n+1 , and then any hyperedge with cardinality less than m can be augmented to a multi-set with m elements. For instance, when m > 3, a hyperedge with vertices {1, 2, 3} can be augmented to {{1, 2, 3, n + 1, . . . , n + 1}} with m − 3 null vertices, whereas another hyperedge with vertices {1, . . . , m} stays the same without any additional null vertex. It is clear that H is converted to an equivalent m-uniform multi-hypergraph, where each hyperedge is a multi-set with cardinality m and only the null vertex v n+1 is allowed to appear multiple times in a hyperedge. With slight abuse of notation, we still denote this m-uniform multi-hypergraph as H. A simple illustration of this augmentation step is displayed in the right panel of Figure 1.
We remark that when H is a uniform hypergraph, the augmentation step and the subsequent special treatments of the null vertex are not necessary. For ease of presentation, we assume H is a non-uniform hypergraph hereafter, and the upcoming proposed model can be adapted to uniform hypergraph with slight modification. As such, we use the mth-order Apparently, a necessary condition for a i 1 ...im = 1 is δ n+1 i 1 ...im = 0. It is clear that this augmentation step, converting a non-uniform hypergraph to a uniform multi-hypergraph, is critical to make downstream analyses more statistically and computationally tractable.
Suppose there are K potential communities within the hypergraph, and let ψ : [n] −→ [K] be the community assignment function, then ψ(i) = s if vertex v i belongs to the s-th community. We also write ψ i = ψ(i) for short. Note that the null vertex v n+1 does not belong to any community, and thus we set ψ n+1 = K + 1 for formality. Denote Z = (z ij ) ∈ R (n+1)×(K+1) to be the corresponding community membership matrix with z ij = 1 if j = ψ i and 0 otherwise. It is obvious that z i,K+1 = 1 only when i = n + 1.

Hypergraph embedding
Given the m-uniform multi-hypergraph H from the augmentation step, we let P where the sparsity factor s n that may vanish with n is introduced to accommodate sparse networks. The modified logit transformation (1) implies that p i 1 ...im = s n (1+e −θ i 1 ...im ) −1 , and the number of hyperedges is of order O p (s n n m ) if Θ lies in a compact subset of R (n+1)×...×(n+1) .
We remark that many frequentist analysis of network models multiplies the network underlying linking probability by a decaying sparsity coefficient to accommodate sparse networks, such as the hypergraph stochastic block model (Ghoshdastidar and Dukkipati, 2017a,b) and hypergraph degree-corrected block model (Ke et al., 2021). Yet this treatment may not be directly applied to latent space model with standard logit transformation, since tiny value of p i 1 ...im implies that θ i 1 ...im will be pushed towards −∞, and thus the entries of α may diverge to ±∞, leading to unstable numerical performance in estimating α.
We now turn to embed the hypergraph into an r-dimensional Euclidean space with 2 ≤ r n, where each vertex is represented by an r-dimensional vector α i for any i ∈ [n]. The embedding dimension r here is allowed to diverge with n. For the null vertex, we simply set α n+1 = r −1/2 1 r , the r dimensional vector with every entry r −1/2 . Let α = (α 1 , ..., α n+1 ) T be the embedding matrix. We consider the following hypergraph embedding model (HEM), where I ∈ R r×...×r is the mth-order identity tensor of dimension r. Clearly, for any i 1 , ..., i m ∈ [n + 1], model (2) assumes that the information contained in θ i 1 ...im can be fully captured by the embedding matrix α in that (2) does not have any probability interpretation when δ n+1,ord i 1 ...im = 1, and is thus inconsequential.
The proposed HEM is flexible and general, and includes the celebrated hSBM (Ghoshdastidar and Dukkipati, 2017a) as its special case. Particularly, taking α = ZC, HEM which becomes an hSBM with membership matrix Z ∈ {0, 1} (n+1)×(K+1) and a transformed core probability tensor K+1) . Yet, HEM is more flexible as it naturally accommodates heterogeneity among vertices by allowing the embeddings or spectral embeddings of vertex within a community to fluctuate in all directions. It is worth pointing out that such accommodation of heterogeneity is more general than that in hDCBM (Ke et al., 2021;Yuan et al., 2021), which requires spectral embeddings of vertices within a community to lie along the same direction and only differ by magnitudes.
It is also interesting to remark that HEM is equivalent to assuming that the super symmetry tensor Θ has a symmetric CP decomposition (Kolda and Bader, 2009) where α ·j is the j-th column of α and • stands for the vector outer product.
Furthermore, we define the symmetric rank of an mth-order tensor Θ of dimension n + 1 over R as rank(Θ) = min{R : Θ = R r=1 u r • ... • u r , u r ∈ R n+1 }. Therefore, it is assumed that Θ has symmetric rank at most r. We also remark our definition of symmetric rank is slightly different from the one that defined in literature (Kolda and Bader, 2009;Robeva, 2016) by requiring the embedding vectors to be real.
More importantly, HEM is identifiable if Θ has symmetric rank r and m ≥ 3, as the factorization (2) is unique up to column permutations of α (Sidiropoulos and Bro, 2000).
When Θ has symmetric rank r but m = 2, HEM reduces to the latent space model for graph networks (Hoff et al., 2002) and the factorization (2) is unique up to any orthogonal transformation of α. Note that column permutations or general orthogonal transformations are isometric linear transformation of the rows of α, the community structure encoded in α always remains unchanged.

Penalized log-likelihood objective
With the factorization of Θ in (2), the negative log-likelihood function of H becomes where ϕ(n, m) = m k=1 n k is the number of potential hyperedges with δ n+1,ord We next equip L(α; A) with a novel penalty term to enhance the feasibility of computation and hypergraph community detection. This leads to the proposed regularized cost function, where λ n is a positive tuning parameter and is introduced to encourage the community structure encoded in α. Herein, Γ is the set of all possible community membership matrix. That is, for any Z ∈ Γ ⊂ R (n+1)×(K+1) , each row of Z contains exactly one 1 with all other entries being zeros, and Z i(K+1) = 1 if and only if i = n + 1. It is clear that the embeddings of vertices with similar linking patterns will be pushed towards the same center, and thus close to each other in the embedding space, leading to the desired community structure in H. A similar regularization term has been employed in Tang et al. (2020) for individualized variable selection and in Zhang et al. (2021) for community detection in directed networks. In the sequel, with Θ = I × 1 α... × m α, we use L λ (Θ; A) and L λ (α; A) interchangeably, as convenience dictates.
We develop an alternative updating scheme to minimize (3). Particularly, given Z (t) and C (t) at step t, α can be updated by solving ..(n+1) = 0, and 0 otherwise. We then update the first n rows of α, which is denoted as α 1:n , along its gradient, α , where η t > 0 is the learning rate at step t + 1, and Herein, * is the Hadamard product (entry-wise product) between two tensors, and T * ∆, I × 2 α × 3 ... × m α {2,...,m} ∈ R (n+1)×r is the tensor inner product between T * ∆ and I × 2 α × 3 ... × m α with respect to the second, ..., m-th modes. Specifically, the (i, j)-th entry Next, given α (t+1) , the sub-optimization task now becomes min Z∈Γ,C∈R (K+1)×r Clearly, it resembles the K-means formulation for α 1:n , and thus a standard K-means algorithm can be employed to solve for Z and C.
As computational remarks, the alternative updating algorithm is guaranteed to converge to a stationary point, and its computational complexity is of order O κ 2 (n m r + κ 1 Knr) , where κ 1 is the number of iterations for the K-means algorithm and κ 2 is the number of iterations for the gradient descent step. Note that low rank tensor approximation tends to be computationally expensive and easy to get trapped in non-informative local minima (Arous et al., 2019). Based on our limited numerical experience, a warm initialization of α (0) can greatly help with the numerical convergence. In all the numerical examples, we initialize α (0) with a higher-order singular value decomposition algorithm (HOSVD;De Lathauwer et al., 2000). We also suggest to set the embedding dimension r = K in practice, similar suggestions were also made for some spectral-clustering-based algorithms (Ghoshdastidar and Dukkipati, 2017a). When K is unknown, we can follow the procedure in Ke et al.

Asymptotic theory
This section establishes some theoretic results to quantify the asymptotic behavior of the proposed HEM method in estimating the underlying transformed probability tensor Θ as well as detecting community structure in a general hypergraph.
The following large deviation inequality is derived to quantify the asymptotic behavior of L λ (Θ; A) in the neighborhood of Θ * defined by e L (Θ, Θ * ).

Proposition 1 gives an intermediate result for establishing estimation consistency in The-
orem 1, and it assures that any Θ such that L λ (Θ; A) ≤ L λ (Θ * ; A) shall lie in the neighborhood of Θ * with high probability. By the condition that nrϕ −1 (n, m) −1 n log −1/2 n ≤ c 1 , the fastest order of n can be set as n = nr log n ϕ(n,m) , which is governed by the network size ϕ(n, m) and the number of parameters nr in HEM. It is interesting to remark that λ n appears to have no effect on the order of n , as long as it satisfies the condition λ n J(α * ) ≤ 1 2 n . We are now ready to establish the consistency of Θ = I × 1α × 2 ... × mα , withα being the estimate from Section 3.3. Let p(y; θ) be the density of a Bernoulli random variable with parameter p = s n (1 + exp(−θ)) −1 . Then the discrete Hellinger distance between p(y; θ) and p(y; θ * ) is defined as d(θ, θ * ) = p 1/2 − (p * ) 1/2 2 + (1 − p) 1/2 − (1 − p * ) 1/2 2 1/2 , and the deviation between Θ from Θ * can be evaluated by the averaged squared Hellinger distance, Theorem 1. Under the assumptions in Proposition 1, for any Θ with L λ (α; A) ≤ L λ (α * ; A), we have P D 2 ( Θ, Θ * ) ≥ n ) ≤ 2 exp − c 2 ϕ(n, m) n , Theorem 1 shows that a reasonably good solution Θ is guaranteed to converge to Θ * at a fast rate, which depends on the centrality of the community structure encoded in α * , the network size and number of parameters via n , and the network sparsity factor s n . The condition L λ (α; A) ≤ L λ (α * ; A) shall be satisfied by the solution obtained in Section 3.3, when the estimation algorithm is initialized by some value in a small neighborhood of Θ * .
The consistency result in Theorem 1 holds true with O(rn 1−m log n) ≤ n s n . If we further set r < log n, this yields a slightly weaker sparsity assumption than that in Ke et al. (2021) and Ghoshdastidar and Dukkipati (2017a,b), where the smallest sparsity factor is of the order n 1−m (log n) 2 for some fixed K.

Consistency in community detection
We now turn to establish the consistency of community detection for general hypergraphs.
Let ψ * : [n] −→ [K] be the true community assignment function, andψ be the estimation counterpart induced byα. Formally,ψ i = arg min k∈ [K] ||α i − C k || 2 , where C = arg min C min Z∈Γ ||α − ZC|| F , subject to C K+1 = r −1/2 1 r . The community detection error ofψ can be evaluated by the minimum scaled Hamming distance betweenψ and ψ * under permutations, which is defined as where 1{·} is the indicator function and S K is the symmetric group of degree K. Clearly, it measures the minimum fraction of vertices that are misclassified byψ under permutation, andψ is a consistent estimator of ψ * if err(ψ * ,ψ) goes to zero with probability tending to 1. Such a scaled or unscaled Hamming distance has become a popular metric in quantifying the performance of community detection (Ghoshdastidar and Dukkipati, 2017b,a;Ke et al., 2021;Jing et al., 2021;Lee et al., 2021).
Let N * k = {i : ψ * i = k} be a true network community with cardinality n k and C * ∈ R (K+1)×r be the associated community embedding center matrix in the sense that The following assumptions are made to ensure that communities within the hypergraph networks are asymptotically identifiable.
Assumption A. There exists a constant c 3 > 0 such that where B * k is the k-th sub-tensor of B * by fixing the first index as k, and γ n may converge to 0 with n.

Assumption B.
There exists a constant c 4 such that max k n k ≤ c 4 min k n k .
Assumption A is an identifiability assumption, which assumes that the true communities are well separated as n grows, and is crucial to the feasibility of community detection. It is interesting to remark that a signal-to-noise ratio is used in Yuan et al. (2021) to characterizes the separability among communities under hSBM. A similar community separation assumption can be found in Lei et al. (2019) for multi-layer network model. Assumption B assures the communities are well defined and will not degenerate asymptotically. This assumption is mild and satisfied when the vertex community memberships come from a multinomial dis-tribution. The same assumption can be found in Ke et al. (2021), and a relatively stronger assumption can be found in Chien et al. (2019) assuming equal community sizes.
Theorem 2 assures that the community structure in a general hypergraph can be consistently recovered by the proposed HEM method. The consistency result holds true for diverging K as long as it does not diverge too fast. Furthermore, Theorem 2 requires that λ n cannot be too small, whereas Theorem 1 requires λ n to be sufficiently small. Combining these two gives a proper interval for the order of λ n to assure consistency in both network estimation and community detection.
As a theoretical example, consider a hypergraph network with J(α * ) ≤ 2 n s 2 n log(s −1 n ) −1 and r, γ n and K are of the constant order. With λ n = 1 2 −1 n s 2 n log(s −1 n ), Theorem 1 implies that n is of the order n 1−m log n, and Theorem 2 further implies that err(ψ * ,ψ) = O p (log n/(n m−1 s n )), which matches up with the error rates in Ghoshdastidar and Dukkipati (2017a,b) and Ke et al. (2021). To ensure the community detection consistency, Theorem 2 requires that s n log n/n m−1 , which is slightly weaker than the sparsity requirement in Ghoshdastidar and Dukkipati (2017a,b) and Ke et al. (2021). We also remark that under homogeneous hSBM, where the probability of any m vertices forming a hypergraph is p if they are from the same community and q otherwise, both error rates and sparsity requirement may be improved (Ahn et al., 2018). However, such results highly rely on the restrictive homogeneous hSBM, and it remains unclear whether they can be extended to heterogeneous hSBM or the even more general HEM.

Numerical experiments
We evaluate the performance of the proposed HEM method on a variety of synthetic and real-life non-uniform hypergraph network data. We compare its performance with some existing non-uniform hypergraph community detection methods in literature, including Tensor-SCORE (Ke et al., 2021), spectral hypergraph partitioning (SHP; Ghoshdastidar and Dukkipati (2017a)), and weighted projection to graph method (WPTG; Kumar et al. (2021); Ghoshdastidar and Dukkipati (2015a)). Tensor-SCORE is designed for uniform hypergraph community detection and can be extended to non-uniform hypergraph by representing nonuniform hypergraph as a collection of uniform hypergraphs or by adding multiple null vertices to convert the non-uniform hypergraph to a uniform one as in Ouvrard et al. (2021). We abbreviate the former extension as TS-1, while the latter extension as TS-2. SHP converts a non-uniform hypergraph to an incident matrix, and then maximizes the hypergraph associativity or minimizes the normalized hypergraph cut. WPTG represents the non-uniform hypergraph by a weighted adjacency matrix, and then standard graph community detection methods such as the spectral clustering, SCORE (Jin, 2015) and modularity maximization algorithms can be employed.
Both HEM and Tensor-SCORE involve some tuning parameters, which can be optimally determined by some data-adaptive selection criteria, including the stability criteria (Wang, 2010) or the network cross validation (Li et al., 2020a). Yet such data-adaptive selection schemes can be computationally expensive. Alternatively, in our numerical experiments, we follow the treatment in Ke et al. (2021) for the tuning parameters in Tensor-SCORE, and set λ n = 10 −4 /n for HEM to prevent J(α) from vanishing too fast. We scale λ n properly in order to ensure more information can be learned from the network likelihood at the early iterations in the computing algorithm. The numerical performance of all the methods is assessed by the average scaled Hamming error in (6).

Synthetic networks
We consider two scenarios of synthetic networks for numerical comparison.
Scenario 1: The non-uniform hypergraph networks are generated from the HEM model in (1) and (2). We vary the number of vertices n ∈ {300, 400, 500}, the sparsity factor s n ∈ {0.4, 0.2, 0.1, 0.05, 0.025}, and set the range of all the hypergraphs to be m = 3 with r = K = 2. First, we generate the community centers C * ∈ R (K+1)×r with C * k ∼ N r (0 r , I r ) for k ∈ [K] and the last row C K+1 = r −1/2 1 r , where I r is the r-dimensional identity matrix.
Next, the vertex community memberships ψ * i , i ∈ [n], are generated from the multinomial distribution with parameters 1 K 1 K indicating that the communities are approximately of equal sizes. The community membership matrix Z * ∈ {0, 1} (n+1)×(K+1) can be constructed accordingly. After that, the hypergraph embedding matrix is generated as α * = Z * C * + ε.
For both scenarios, the averaged scaled Hamming errors and their corresponding standard errors of various community detection methods over 50 independent replications are reported in Tables 1 and 2.
It is evident that HEM yields the smallest community detection error among all the methods in both scenarios. The performance of HEM and Tensor-SCORE is much better than that of SHP and WPTG, mainly due to the fact that SHP and WPTG need to convert the hypergraph adjacency tensor to matrix and thus suffer from information loss. Further, Table 1: The averaged community detection errors and its standard errors over 50 independent replications of different methods on the non-uniform hypergraph networks generated from Scenario 1.  the performance of TS-2 appears to be better than TS-1, suggesting that converting a nonuniform hypergraph to a uniform one by adding null vertices can be a better data processing approach than decomposing a non-uniform hypergraph into a collection of uniform hypergraphs of different range. It is also interesting to note that HEM is fairly robust to the hypergraph network sparsity and the community imbalance, whereas the performance of other competing methods can be substantially affected.
To examine the network estimation accuracy, the averaged estimation errors of HEM, measured by n −m/2 || Θ − Θ * || F , and their corresponding standard errors are reported in Table 3. Note that other competing methods solely focus on community detection, and thus do not produce estimate of Θ * . Clearly, the estimation error of HEM becomes smaller as the number of vertices increases, the network becomes denser or the communities become more balanced.

Two real-life hypergraph networks
We now apply the proposed HEM method to analyze two real-life hypergraph networks, including the Medical Subject Headings (MeSH) hypergraph network (Ke et al., 2021)  there are a total of 1,375 hyperedges left, consisting of 297, 883, 174, 20 and 1 hyperedges of size 1, 2, 3, 4, and 5, respectively. We also remove those 21 hyperedges of size greater than 3 and all the hyperedges of size 1 due to their lack of information about latent community structure. After the pre-processing, we obtain a hypergraph of range 3 with 281 vertices and 1,057 hyperedges of size 2 or 3, where 180 vertices come from C04 and the other 101 vertices come from C10.
The SPECT network (https://archive.ics.uci.edu/ml/datasets/spect+heart) contains the SPECT images of 267 patients, where each image has been processed to 44 categorical fea-tures to discriminate abnormal patients from the normal ones. To construct the hypergraph network, each patient is represented as a vertex, and for a possible value of each feature, we construct a hyperedge that contains all the patients sharing the particular feature value.
Such a hypergraph construction method has been studied by Schölkopf et al. (2007) and Ghoshdastidar and Dukkipati (2017a). After deleting the replication hyperedges, there are a total of 1,483 hyperedges with sizes verying from 1 to 36. We only consider those hyperedges of size ranging from 2 to 6, and for hyperedges with sizes greater than 3, we convert them to all possible 3-cliques, leading to multiple hyperedges with size 3. Replication hyperedges and isolated vertices are further deleted. After the pre-processing, we obtain a hypergraph with 264 vertices and 2,950 hyperedges, where 211 vertices come from the abnormal patients and the other 53 ones come from the normal patients.
We perform different community detection methods on both hypergraph networks. As suggested in Ke et al. (2021), we set K = 6 for the reg-HOOI algorithm in Tensor-SCORE, and thus we also set the embedding dimension r = 6 in HEM for fair comparison. The network sparsity factor in HEM is estimated by hyperedge density; that is, s n = 1057 ×  Table 4.

Conclusion
This article proposes a novel community detection method on general hypergraph networks.
The proposed method is built upon a tensor-based hypergraph embedding model, which consists of a network augmentation step and an network embedding step. The resultant negative log-likelihood function is equipped with a new regularization term to encourage community structures among the embedding vectors. The proposed method is supported by various numerical experiments and asymptotic consistency in terms of community detection. Particularly, the theoretical results can be established for very sparse hypergraph network with link probability of order s n n 1−m log n. It is worth pointing out that the proposed community detection method can be extended to various scenarios, such as multilayer hypergraph networks or communities with mixed memberships, which is under further investigation.