{'fraction_non_alphanumeric': 0.028380429991674422, 'fraction_numerical': 0.007003281257652187, 'mean_word_length': 5.021675022117369, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Topical Keyphrase Extraction with Hierarchical Semantic Networks

Topical keyphrase extraction is used to summarize large collections of text documents. However, traditional methods cannot properly reflect the intrinsic semantics and relationships of keyphrases because they rely on a simple term-frequency-based process. Consequently, these methods are not effective in obtaining significant contextual knowledge. To resolve this, we propose a topical keyphrase extraction method based on a hierarchical semantic network and multiple centrality network measures that together reflect the hierarchical semantics of keyphrases. We conduct experiments on real data to examine the practicality of the proposed method and to compare its performance with that of existing topical keyphrase extraction methods. The results confirm that the proposed method outperforms state-of-the-art topical keyphrase extraction methods in terms of the representativeness of the selected keyphrases for each topic. The proposed method can effectively reflect intrinsic keyphrase semantics and interrelationships.


Introduction
Because of the rapid increase in the number of digital documents, systemized compression of big unstructured text data has become inevitable for efficient retrieval of useful information [1,2]. Organized knowledge of text data patterns can assist those who lack an overall idea of the domain. Many academic organizations and mass media provide category information that broadly explains the content of documents. However, this information lacks 1 Abbreviations: HSN: Hierarchical semantic network, LDA: Latent Dirichlet allocation, TF-ITF: Term frequency-inverse topic frequency, TPR: Topical PageRank, TR: TextRank, WPR: Weighted PageRank directly associated keyphrases that provide a clear guideline, and obtaining them is a major challenge [3]. Automatic keyphrase extraction methods are expected to provide guidelines for extracting a set of phrases that preserve the main theme of a document [4]. Selecting the appropriate keyphrases is essential in domains that require an effective and fast understanding of large text collections [5].
However, document data tend to involve a large number of distinct topics. Therefore, without adequate topic information, it is difficult to extract keyphrases that represent the entire context. Thus, topical keyphrase extraction methods were developed, where the topic is considered in keyphrase extraction. With topical keyphrase extraction methods, searchers encounter many groups of keyphrases in which each group symbolizes a single topic in the document. In each group, the keyphrases represent the group topic. These are called topical keyphrases and are expected to provide a better overview of the entire document data. Numerous topical keyphrase extraction methods have been developed; they use clustering techniques [6], language modeling [4], statistical modeling, and network-based approaches. Statistical models select topical keyphrases through the inferred topic-word probability or a specific termfrequency rule. These models are primarily intended for statistical interpretation of term specificity [1,7]. However, they cannot capture the underlying meaning of keyphrases because they are primarily based on the computation of the frequency of keyphrases.
Network-based topical keyphrase extraction methods attempt to determine the semantic relationships of keyphrases. These methods initially construct a phrase network and rank its phrases according to their salience. A phrase network is composed of nodes and edges. Nodes correspond to phrases, and edges represent the relatedness of the nodes they connect. To compute node relatedness, the cooccurrence frequency of nearby words is typically used. For phrase-based TR, a node is directed to the another node when the corresponding word is positioned before other words in a word sequence [8]. For TPR, two nodes are linked if their corresponding words co-occur in a W-sized window. The edge directions are determined by the first word pointing to every other word in the same window, and the edge weights are simply set to the co-occurrence counts within a window [9]. In both methods, PageRank, a network centrality measure, determines the representativeness of a keyphrase [9].
Although phrase evaluation in previous network-based ranking approaches can recognize elementary keyphrase relationships, hierarchical relationships cannot be identified. To address this limitation, we propose an HSN to extract topical keyphrases through centrality measures. Identifying hierarchical keyphrase relationships is motivated by [10], where it is considered important in obtaining the main themes of document data by detecting the phrase in a group that contains the most general information and thus summarizes the group content. For example, the phrase "machine learning" is more general than "supervised learning" in documents concerning machine learning algorithms that include the concept of supervised learning. The hierarchical relationships are obtained by a frequency rule called the "subsumption rule" [10].
In this study, we propose an HSN that selects the most representative topical phrases by considering their complex and hierarchical relationships. Traditional methods cannot explicitly detect such complex and intrinsic relationships, whereas the hierarchical organization in the proposed method enables sorting keyphrases according to their degree of representativeness; thus, we can detect the phrases that best represent each topic within a given set of documents.
The main contributions of this study can be summarized as follows: (1) The proposed method can reflect the hierarchical relationships between phrases by using an association rule in the construction of a hierarchical semantic network. These relationships identify the edges of the network and determine which phrases should be used as nodes. Only nodes with close hierarchical relationships are added to the network, along with adequate edge directions and edge weights.
Unlike other network-based keyphrase extraction methods that use simple co-occurrence rules, the proposed method uses the subsumption rule, which considers the hierarchical relationships between phrases.
(2) We propose topical keyphrase selection by combining the phrase scores by multiple centrality network measures. The representativeness of each node in the phrase network is determined by its inflowing edges and edge weights. Thus, we integrate some centrality measures that can evaluate the nodes according to their inflowing edges. Because these centrality measures reflect the phrase hierarchy, the most highly evaluated phrases can best represent the topics of a document. (3) To demonstrate the usefulness and applicability of the proposed method, we use real-world large text data for comparison with existing keyphrase extraction methods in terms of effectiveness in selecting representative topical keyphrases. The results confirm that the proposed method outperforms the others.
The remainder of this paper is organized as follows. In Section 2, we review existing keyphrase extraction approaches associated with the present study. In Section 3, we present the details of the proposed HSN. Section 4 presents and discusses the experimental results and their evaluation. Finally, Section 5 concludes the paper.

Related Work
The use of supervised and unsupervised approaches in topical keyphrase extraction has attracted significant attention. Various supervised learning algorithms, which are often treated as binary classification algorithms, were trained on documents annotated with keyphrases to determine whether a candidate phrase is a keyphrase or not [5]. However, recasting keyphrase extraction as a classification problem suffers from the weakness of treating keyphrases independently of one another in the training process [5,11]. Consequently, it is difficult to compare them and select the keyphrase that best represents the entire set of documents.
To compensate for the limitations of supervised approaches, unsupervised approaches have been introduced. For example, a key cluster algorithm was proposed to cluster semantically similar candidate phrases using Wikipedia and co-occurrence-based statistics [5,6]. This method selects the centroid of each topic cluster as a topical keyphrase. However, its drawback is that each topic is given equal importance. In practice, there could be topics that are unimportant and should not have keyphrases representing them [5]. To extract all candidate keyphrases from important topics through Wikipedia, a community cluster algorithm was proposed [5]. However, because the algorithm depends on Wikipedia-based knowledge, it cannot reflect specialized topics in specific document data.
Keyphrase ranking is a different methodology in which candidate phrases are scored and only those with the highest scores are selected as representative topical keyphrases. TF-ITF is a variant of the traditional term frequency-inverse document frequency that is used for statistically ranking the specificity of index terms and document descriptions [3,7]. Accordingly, the weight of a keyword is the product of the number of occurrences in the document (DF) and the rarity value across the entire document collection (IDF). TF-ITF counts the number of words in the topical document collection instead of a single document. The TF-ITF weights are statistical measures for evaluating the degree to which a word is relevant to the corresponding topic. This implies that highly weighted keyphrases are used as representative keyphrases for the topic. A well-known topic model, LDA, can extract keywords based on a topic model that takes documents as input, models them as mixtures of different topics, and discovers word distributions for each topic [12]. The word probabilities can be interpreted as weights for ranking the degree of relevance of the words to the topics. A word with higher probability in the word distribution is considered to be more representative of the topic, compared with other words in the document collection. Although the keywords and keyphrases resulting from the TF-ITF and LDA algorithms may encompass topic-relevant information, they cannot capture the semantic relationships of keyphrases.
Network-based ranking methods are the state-of-the-art in unsupervised approaches [9,13]. The TR, TPR, and Node and Edge Rank (NE-Rank) models provide network measures for ranking candidate words [8,9,14]. TR generates a word network that considers the cooccurrence of words in the neighboring sequence of a document. Each phrase is then scored by a PageRank network measure [8]. The top-ranked words are post-processed to be combined into phrases if they are adjacent in the text; otherwise, they remain as keywords. TPR constructs a phrase network based on the co-occurrence of phrases in the phrase sequence and scores each phrase using a modified PageRank network measure. TPR includes information on word distribution per topic from LDA and thus yields representative topical keyphrases [9]. The NE-Rank algorithm is a combination of the TR and PageRank network measures with edge weight information. It is designed to extract topical keyphrases from short texts such as tweets on Twitter [14]. The three aforementioned network-based ranking methods have limitations regarding the construction of the phrase network. Because phrase relations in the network are implicated by a simple cooccurrence rule applied to neighboring phrases, hierarchical relationships cannot be comprehended. Moreover, TPR induces a limited selection of keyphrases because it directly adopts the LDA word distribution per topic into the PageRank network measure.

Overview of the proposed topical keyphrase extraction method
Network-based topical keyphrase extraction methods consist mainly of four procedures: (1) topical document collection, (2) topical candidate phrase composition, (3) construction of the topical hierarchical semantic network, and (4) evaluation of topical candidate phrases. Steps (3) and (4) are the main contributions of the proposed HSN, which essentially reflects the hierarchical relationships of topical candidate phrases in the construction of the semantic network and phrase evaluation. First, the relationships between topical candidate phrases are identified by a subsumption rule, and an association rule is used to define the nodes and edges of the topical HSN. Then, several centrality network measures are used to evaluate the phrases in the network. Finally, the most highly evaluated topical keyphrases are extracted. Fig. 1 shows an overview of the proposed HSN. To obtain topical keyphrases, we collect the documents associated with the main topics from document data (i.e., topical document collection) and then extract topical keyphrases from each of them. We use LDA to collect the documents in topics. In LDA, the documents are regarded as random mixtures of latent topics that are characterized by distribution over all the words. Each word of document is generated by first sampling a topic from the corresponding topic distribution per document ( ( ) ) and then sampling a word from the corresponding word distribution per topic ( ( ) ). To cover a document, this process is repeated for every word in the document.
( ) and ( ) are parameterized by conjugate Dirichlet priors and . The word probability of is represented as follows: where the number of topics is given by [9,12]. In the proposed method, we set as an asymmetric prior from the corpus for ( ) because this induces a more robust and datadriven model than a symmetric prior [15]. Thus, we obtain the per-document topic distribution ( ) and the per-topic word distribution ( ) . Both are used in the proposed method. After the per-document topic distribution has been determined, the documents are collected by topic as follows: where is a threshold for gathering the documents most related to topic . C is the document collection for topic where ( ) is larger than τ.

Topical candidate phrase composition
From each topical document collection, we extract the most representative candidate phrases. From the LDA model used to obtain the topical document collections, we use ( ) to select the 30 words with the highest probability for each topic. These are selected as LDA keywords for topic . For every , we retrieve all the noun phrases that contain the LDA keywords for . Noun phrases are defined as "Adjective+Noun," "Noun+Noun," and "Adjective/Noun+Noun" forms. Through a linguistic processing step, we identify each word in using part-of-speech tagging [16] and obtain the noun phrases. We discard any noun phrase that includes no LDA keywords. We assume the use of 30 words is generally acceptable because using more words would include redundant information in the topic representation. Noun phrases are specifically selected on the premise that they convey most of the meaning of a text [17]. These are selected as candidate phrases for every topic and are used to construct the HSN in the next section.

Construction of a hierarchical semantic network
Among the candidate phrases for topic , we attempt to select the most comprehensive and representative. It is conceivable that the hierarchical organization of the candidate phrases would facilitate this selection. The principles of hierarchical organization of phrases are as follows: (1) Every phrase in the hierarchy is drawn from the documents and best reflects the topics covered there. (2) The hierarchical organization would be such that an antecedent phrase would refer to a more general concept than its consequent phrase. The concept of antecedent subsumes the concept of consequent [10].
Considering these principles, the practicalities of constructing an HSN for each topic are addressed in the proposed method. The proposed HSN uses candidate phrases that best reflect the topics as components of the network.
Moreover, the subsumption relationships of candidate phrases for each topic define the nodes, edge directions, and edge weights of the networks.

Identifying the nodes
Before determining the nodes, we select only those phrases likely to have a hierarchical relationship. We thus dismiss any redundant phrases unrelated to other phrases in the document. We assumed that phrase pairs that co-occur in the same document are semantically related. To incorporate this idea, the proposed HSN adopts the "support" metric of the association rule, which indicates the proportion of the documents in in which a phrase pair appears [20]. Specifically, the support of phrase pair can be defined as follows: where is a phrase pair from (which is the set of all possible pairs of candidate phrases for topic ), 1 and 2 denote phrases in the pair , and is the number of documents in . Then, we select those phrase pairs whose support exceeds the user-specified threshold. That is, where is the support threshold. The phrase pairs in correspond to the nodes in of : = ( , ) for topic . The implication is that a pair with large support, that is, two phrases that frequently appear simultaneously in a document, has a relatively strong semantic connection.

Defining the edges
After the phrase pairs of have been used to define the nodes of the network, the proposed method identifies (edges and edge directions) of : = ( , ) through the subsumption rule. The nodes may be either antecedent or consequent, and the edges are directed from consequents to antecedents [10]. According to the association rule, the confidence of a phrase pair can be defined as follows [21]: where denotes a phrase pair 1 , 2 ,. Specifically, confidence is the proportion of documents that contain both 1 and 2 among the documents that contain only 1 in . This implies the probability of the presence of 2 when 1 is present. Because the "confidence" metric of the association rule involves the same idea, Equation (5) can be used to assess the subsumption relationships of phrases as follows: where denotes a phrase pair composed of 1 , 2 , and ' denotes the same phrase pair in reverse order (i.e., 2 , 1 ,). If the conditions of Equation (6) are met, 1 is the antecedent and 2 is the consequent. The implication is that 1 , which entails more general meaning, always tends to exist when 2 , which entails less general meaning, exists in a document. Meanwhile, 2 does not always tend to exist when 1 exists in the document. Thus, the edge between the two phrases is directed from 2 to 1 , which implies that 1 contains more general concepts than 2 [10]. Consequently, the edge set of : = ( , ) for topic is defined through the subsumption relationship of phrases. The nodes that do not satisfy Equation (6) are discarded from because their edge directions cannot be determined.

Determining the edge weights
To complete the construction of the HSN, we use the metric "lift," which determines edge weights based on the degree of relationship of two connected nodes. Lift can quantify the subsumption relationship between two phrases; greater lift values indicate stronger relationships because lift refers to the dependency probability of two phrases [20]. When two events are independent of each other, it would be meaningless to draw any relationship from them. The lift of the phrase pair can be defined as follows: and measures the degree to which phrase 1 subsumes phrase 2 . It is used to define = [ℎ , ].

Evaluating candidate phrases
Once we have constructed the HSN for each topic, we measure and rate each phrase through indegree centrality, eigencentrality, and WPR in directed networks [19,22]. We select these three measures among other centrality network measures because they primarily use inflowing edges and directions for properly measuring antecedent nodes in the network. Other measures such as betweenness centrality weigh the salience of a node by its transferability in the shortest path [23]. The followings are definitions and brief explanations of the three measures. The indegree centrality score of node in a directed network can be calculated as [24,25] That is, indegree centrality sums the number of inflowing edges [22]. Therefore, the node with the largest number of inflowing edges is granted the highest score. In the HSN, the phrase that contains the most subsuming information is likely to receive the highest score because the consequent phrase is directed to the antecedent phrase.
In addition, we use eigencentrality score of node , which can be calculated as [25,26] where is the eigenvalue of the adjacency matrix , . Eigencentrality measures the salience of nodes that are recursively related to directly connected nodes [27]. For example, in the HSN, node , which is determined to be representative of a topic, can contribute to the representativeness of node . The application of eigencentrality to the HSN is particularly meaningful because it can globally reflect the hierarchical relationships of phrases.
Finally, we use WPR, which, in addition to edge direction, considers edge weights, when scoring a node. We identify the degree of relationship of candidate phrases using the lift values in Equation (8). The WPR score of node is calculated as [19] where is the damping factor. , is the weight of ( , ), calculated based on the number of inflowing edges of node and the number of inflowing edges of all nodes . , is the weight of ( , ) calculated based on the number of outflowing edges of node and the number of outflowing edges of all nodes . By using WPR, we can estimate the degree of generality or representativeness of the antecedent phrase with respect to its consequent nodes.
After computing the node scores in terms of indegree centrality, eigencentrality, and WPR, we scale each network measure score from zero to one to obtain the scaled score of each node with respect to the other nodes. This is the degree of representativeness of a specific phrase compared with the other phrases. We then select the top phrases according to the three measures and form the set r as follows: where ( ), ( ), and ( ) refer to the ranks of phrases with respect to each measure. From r , we select the phrases that are present in all three measure results and assign it as HSN-Keyphrase. If a phrase is not present in all three measure results, we discard the phrase. Consequently, we consider only the relatively highly ranked phrases with respect to all three network measures.
The following HSN-Score is calculated by summing the scaled scores of the corresponding HSN-Keyphrase.
where ( ), ( ), ( ) are the scores of the phrase (node) . We limit the keyphrases as in Equation (12), to prevent the bias of a network measure score from excessively influencing the HSN-Score. For example, without using , the HSN-Score of the HSN-Keyphrase may be excessively high when the indegree score of a node is overly high and the other two scores are overly low. Finally, by ranking the HSN-Keyphrase using the sorted HSN-Keyscore, we obtain the keyphrases by the proposed scheme. We perform the same iterative process for all candidate phrases in each topic.

Experimental evaluation 4.1. Experimental data
To evaluate the performance of the proposed HSN, we used two datasets from the Scopus and Arxiv databases, which include scientific papers, books, and conference proceedings.
Both datasets concern artificial intelligence, computer science, and machine learning. The Scopus dataset consists of 108,200 abstracts, and the Arxiv dataset consists of 13,176 abstracts. We tested the algorithms in different topical document collections obtained from LDA models using different numbers of topics . The topical keyphrase extraction results could vary, depending on , because the topical keyphrases were generated from the phrases in each topical document collection. We set to 10, 20, and 30. We speculated that if the proposed method outperformed other keyphrase extraction methods regardless of , this would improve its robustness. We set τ , the threshold for generating the topicdocument collection, to 0.95, 0.90, and 0.80. We set the minimum support threshold ( ) to 0.005 to obtain the candidate phrase pairs.

Keyphrase extraction results of the proposed method
This section presents the results of the proposed method tested on the Scopus and Arxiv datasets. Because of space limitations, only the results for Topic 1 of 10 for each dataset are presented. We describe the construction of the HSN for each topic, the ranking and scaling of phrases with respect to different centrality measures, and the final keyphrase extraction. Fig. 2 shows an example of phrase pairs and edges for Topic 1 of 10 from the Scopus and Arxiv datasets. We note that Fig. 2 partly presents the complicated hierarchical semantic network to exemplify how the phrase pairs are connected to construct the network. The nodes refer to phrases, the edge arrows are directions that indicate which phrases are antecedents or consequents, and the edge weights are degree to which the linked phrases are related. After determining the appropriate candidate phrase pairs, edge directions, and edge weights, we constructed the HSN for each topic. Subsequently, we rated each phrase using the three network measures, namely, indegree centrality, eigencentrality, and WPR. After separately computing the node scores in terms of indegree centrality, eigencentrality, and WPR, we scaled each network measure score from zero to one. Then, we selected the top phrases with respect to the three measures. We set to 100 to obtain unbiased results when selecting the HSN-Keyphrase from the network measures. Table 1 shows an example of the network measure results.
From the top phrases, we selected the phrase that simultaneously maximizes the three scores and used it as the HSN-Keyphrase. Moreover, the HSN-Score was calculated by summing the scaled scores of the corresponding HSN-Keyphrase. Table 2 shows an example of the final keyphrase extraction results. For every topic in the document collections obtained using a different number of topics ( 10, 20, 30), we selected the 10 most highly ranked topical keyphrases by each algorithm.

Experimental Evaluation
We performed a blind test to assess the validity of the proposed method [25]. We used Amazon Mechanical Turk (Mturk), an online web-based platform (https://www.mturk.com), for conducting surveys and recruiting randomly selected people to take the blind test. Amazon Mturk is a reliable survey source because its participants are believed to be sufficiently representative and diverse to be involved in research studies [28,29]. The participants in the blind test received the titles most related to the topic as a guideline. One hundred people chosen at random participated in ranking the six algorithms according to their satisfaction with the topical keyphrase sets. Their satisfaction depended on how well each keyphrase set represented and summarized the topic. Among the six algorithms, singlekeyword extraction methods (LDA, TF-ITF) were included to globally evaluate the proposed method. For every topic in the document collections obtained using a different number of topics ( 10, 20, 30), we averaged the rankings of the six algorithms, as determined by the participants. The lowest value represents the highest rank.
The blind test was designed using documents highly related to the topic and six topical keyphrase sets. We selected five documents that had the highest probability of containing the topic from the LDA model and gathered their titles. For instance, an example title set for Topic 1 of 10 from the Scopus dataset is as follows: (1) "Differences between adaptors and innovators in the context of entrepreneurial potential dimensions," (2) "Research for finding relationships between mass media and social media based on agenda setting theory," (3) "Identification of the learning behavior of students for education personalization," (4) "With a little help from my friends: a computational model for the role of social support in mood regulation," and (5) "Lecturer perceptions of impoliteness and inappropriateness in student e-mail requests: a Norwegian perspective." These titles were provided to the participants, who were asked to rate the six topical keyphrase extraction methods. Table 3 shows an example of the six topical keyphrase extraction results. An example title set for Topic 1 of = 10 from the Arxiv dataset is as follows: (1) "Towards effective research-paper recommender systems and user modeling based on mind maps," (2) "Communications and control for wireless drone-based antenna array," (3) "Intent-aware contextual recommendation system," (4) "Probabilistic graphical models for credibility analysis in evolving online communities," and (5) "Explainable recommendation: theory and applications." These titles were provided to the participants to rate the six topical keyphrase extraction methods. Table 4 shows an example of the six topical keyphrase extraction results.  20,30). This demonstrated that the extracted keyphrases by the proposed HSN were better than those by the other five algorithms in various settings. We averaged the ranks by the participants for each topic. (See Appendix A for details.) To facilitate the interpretation of the results, we averaged the average ranks for each topic. Fig. 3 shows the average of the average ranks of the six algorithms in terms of their accuracy with different numbers of topics in the Scopus dataset. The overall user satisfaction results demonstrate that the proposed HSN selected the most meaningful keyphrases to represent the topics. Because the LDA and TF-ITF are primarily unigram-centric, their results have limited semantic capacity. For example, "social" in the TF-ITF results in Table 3 is insufficient to represent Topic 1 of the Scopus dataset. Moreover, although the N-gram-based TF-ITF algorithm ensures the "semantical richness" of the extracted keyphrases, it suffers from some inherent problems that result in its inability to detect the most representative topical keyphrases. Both TF-ITF algorithms depend mostly on phrase frequency. For example, in Table 3, "college student" is ranked fourth in the N-gram-based TF-ITF results, but "collaborative learning" is ranked tenth. Obviously, the keyphrase "collaborative learning" is a more general concept than "college student" is, and therefore it should be regarded as the main theme of the entire context. It is conceivable that a superior topical keyphrase extraction method should distinguish the two keyphrases and select "collaborative learning." Therefore, the results of TF-ITF algorithms are often biased, and higher phrase frequency favors them.
The performance of TR is also worse than that of the proposed HSN. Because the algorithm assumes that relevant phrases are neighbors, it creates adjacent network edges. This sequence information derives a strong relevance only between nearby words and hence reflects only a local portion of the document. When PageRank is applied to the network, it outputs only similar nearby words. Consequently, the results are narrow and unrepresentative of the full scope of the documents. For example, in Table 3, TR extracts "modeling," "modelling," and "modeler" as its top-ranked results. However, these three words appear to have been extracted only because they are lexically distinct, and no extraction was made for keyphrases that globally represent the entire topic. Moreover, TR cannot distinguish hierarchical relationships among the phrases; therefore, it yields keyphrases that are related to a redundant concept such as "model." The performance of TPR is also considered poor compared with that of the proposed HSN. As in the case of TR, the edge weights of the nodes (phrases) are based on phrase co-occurrence within a word sequence window. This prevents the algorithm from extracting global keyphrases related to the topic. Moreover, TPR adds the word probability for a topic from the LDA to the PageRank scoring formula. Directly adding such a term to the scoring function may reduce the accuracy of the PageRank scoring formula because the cumulative sum of phrase scores is calculated. Thus, when the per-topic word probability is explicitly added to the scoring formula, the score of specific keyphrases is exponentially increased, yielding biased results. For example, in Table 3, the most highly ranked word of the LDA results is "social." Because the TPR is directly influenced by the probability of the word "social," its results all include it. Such results are unrepresentative of the entire topic. By contrast, the results do not include keyphrases such as "human perspective" and "learning environment." This implies that excessive weighting of a word probability term can hamper a ranking algorithm in its selection of the most general concepts. Fig. 4. shows the average ranks of the six algorithms with different number of topics ( 10, 20, 30) in terms of their accuracy in the Arxiv dataset. We averaged the ranks by the participants. (See Appendix A for details.) To facilitate the interpretation of the results, we averaged the average ranks for each topic. Fig.4. shows the average of the average ranks of the six algorithms in terms of their accuracy for different numbers of topics in the Arxiv dataset.  Topics   LDA  TF-ITF  N-gram TF-ITF  TR  TPR  HSN (proposed) example, in the LDA and TF-ITF results in Table 4, keywords such as "user" and "recommendation" do not carry sufficient information to represent Topic 1 of the Arxiv dataset. Moreover, N-gram TF-ITF keyphrase results, such as "social dilemma" and "cold start," have no content that can summarize Topic 1. This suggests that considering only phrase frequency may prevent the extraction of the most representative phrase from the topical document collection. Although TR and TPR yield results that reflect the semantic relationships between phrases, they adopt nonhierarchical relationships. For example, in Table 4, the TR produces several redundant words, such as "model," "modeling," "use," "useful," "base," and "user," as its top extracted results. These words not only have limited semantic capacity but also lack the lexical abundance to be regarded as representations of the topical documents.
It is conceivable that a keyphrase extraction model should properly select the core words that can best represent the topic, and it should discard the others. However, the results of TPR in Table 4 do not contain any critical information that can summarize the topic; rather, there are numerous phrases including the word "network." This bias seems to derive from the modified PageRank scoring formula, which is explicitly influenced by the probability of the word "network." Thus, as in the Scopus dataset, the proposed HSN exhibits superior performance compared with the other algorithms in the Arxiv dataset.

Conclusions
Manually organizing a vast number of collected documents and determining their main theme is a tedious task. Topical keyphrases can serve as guidelines so that users may briefly overview the content of such a collection. Several studies have been conducted to develop an efficient topical keyphrase extraction method; however, existing methods use simple term frequency rules and nonsemantic findings. To overcome this limitation, we use the HSN, which allows us to quantify direct, indirect, and integral relations among topical keyphrases [25,30].
From the experimental results and analysis, we highlighted the following findings and practical implications. The underlying principles of TF-ITF and N-gram-based TF-ITF consider only term frequency from a topical bag-of-words without forming any semantic relationships. Thus, they are unlikely to capture the general meaning of a text. The proposed method overcomes this limitation by extracting N-gram keyphrases and using a hierarchical semantic network. Consequently, the most representative phrases can be identified. The major limitation of TR and TPR is the use of a term counting processes in a word sequence of limited size. Such proximity relationships produce meaningless phrases. On the contrary, the proposed HSN emphasizes the hierarchical semantics of each keyphrase using three network measures. Thus, the proposed HSN ensures the quality, diversity, and representativeness of the extraction results, which can well represent the given documents.
In future work, we would like to apply the topical keyphrase extraction methods to document summarization tasks. Because the topical keyphrase results capture the most general concept of each topic, it would be interesting to aggregate the keyphrases into valid sentences. The sentences could then be compounded to become a summary. Moreover, we would like to assign more weight to a specific network measure when detecting semantic relationships. As these approaches confer different advantages, meticulously applying them could certainly improve the proposed method.  Tables A1, A2, and A3 show the average ranking that the participants assigned to the six algorithms in terms of their accuracy in representing the topics in the Scopus dataset. Boldface entries represent the highest average rank awarded by the 100 participants, and the values in the brackets indicate the standard deviation of the ranks. For statistical comparison, we conducted the Wilcoxon Ranksum test (a nonparametric test) to compare the ranks. We rejected the null hypothesis that the ranks of each algorithm and the HSN do not significantly deviate if the p-value was lower than 0.05. Table A1. Comparison of six topical keyphrase extraction algorithms. Each value indicates the average rank by the participants for 10 in the Scopus dataset. The average rank of each algorithm is listed in the bottom row. Each value in a bracket indicates the standard deviation of the ranks. Each * tagged on the average rank represents statistically significant difference of the ranks between the corresponding algorithm and the proposed HSN, with the p-value being lower than 0.05  Table A2. Comparison of six topical keyphrase extraction algorithms. Each value indicates the average rank by the participants for 20 in the Scopus dataset. The average rank of each algorithm is listed in the bottom row. Each value in a bracket indicates the standard deviation of the ranks. Each * tagged on the average rank represents statistically significant difference of the ranks between the corresponding algorithm and the proposed HSN, with the p-value being lower than 0.05. Tables A4, A5, and A6 show the average rankings that the participants assigned to the six algorithms in terms of their accuracy in representing the topics in the Arxiv data set. Boldface entries represent the highest average rank. Table A4. Comparison of six topical keyphrase extraction algorithms. Each value indicates the average rank by the participants for 10 in the Arxiv dataset. The average rank of each algorithm is listed in the bottom row. Each value in a bracket indicates the standard deviation of the ranks. Each * tagged on the average rank represents statistically significant difference of the ranks between the corresponding algorithm and the proposed HSN, with the p-value being lower than 0.05.  Table A5. Comparison of six topical keyphrase extraction algorithms. Each value indicates the average rank by the participants for 20 in the Arxiv data set. The average rank of each algorithm is listed in the bottom row. Each value in a bracket indicates the standard deviation of the ranks. Each * tagged on the average rank represent the statistically significant difference of the ranks between the corresponding algorithm and the proposed HSN, with the p-value being lower than 0.05.  Table A6. Comparison of six topical keyphrase extraction algorithms. Each value indicates the average rank by the participants for 30 in the Arxiv data set. The average rank of each algorithm is listed in the bottom row. Each value in a bracket indicates the standard deviation of the ranks. Each * tagged on the average rank represents statistically significant difference of the ranks between the corresponding algorithm and the proposed HSN, with the p-value being lower than 0.05.  Figure B shows one of the survey questions that were provided to the participants. It consists of five topic-relevant research paper titles and the keyphrase sets obtained by the keyphrase extraction method. Each keyphrase set was produced from the same topical document collection. Figure B. Survey example for 10 in the Scopus dataset. Each participant was asked to rate the six keyphrase sets by LDA, TF-ITF, N-gram TF-ITF, TR, TPR, and the proposed HSN according to the five research paper titles.