{'fraction_non_alphanumeric': 0.03836993913733792, 'fraction_numerical': 0.012731175208020935, 'mean_word_length': 4.559333115397188, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 7, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
An Entity-Topic Model for Entity Linking

Entity Linking (EL) has received considerable attention in recent years. Given many name mentions in a document, the goal of EL is to predict their referent entities in a knowledge base. Traditionally, there have been two distinct directions of EL research: one focusing on the effects of mention's context compatibility, assuming that "the referent entity of a mention is reflected by its context"; the other dealing with the effects of document's topic coherence, assuming that "a mention's referent entity should be coherent with the document's main topics". In this paper, we propose a generative model -- called entity-topic model, to effectively join the above two complementary directions together. By jointly modeling and exploiting the context compatibility, the topic coherence and the correlation between them, our model can accurately link all mentions in a document using both the local information (including the words and the mentions in a document) and the global knowledge (including the topic knowledge, the entity context knowledge and the entity name knowledge). Experimental results demonstrate the effectiveness of the proposed model.


Introduction
Entity Linking (EL) has received considerable research attention in recent years (McNamee & Dang, 2009;Ji et al., 2010). Given many name mentions in a document, the goal of EL is to predict their referent entities in a given knowledge base (KB), such as the Wikipedia 1 . For example, as 1 www.wikipedia.org shown in Figure 1, an EL system should identify the referent entities of the three mentions WWDC, Apple and Lion correspondingly are the entities Apple Worldwide Developers Conference, Apple Inc. and Mac OS X Lion in KB. The EL problem appears in many different guises throughout the areas of natural language processing, information retrieval and text mining. For instance, in many applications we need to collect all appearances of a specific entity in different documents, EL is an effective way to resolve such an information integration problem. Furthermore, EL can bridge the mentions in documents with the semantic information in knowledge bases (e.g., Wikipedia and Freebase 2 ), thus can provide a solid foundation for knowledge-rich methods.

Figure 1. A Demo of Entity Linking
Unfortunately, the accurate EL is often hindered by the name ambiguity problem, i.e., a name may refer to different entities in different contexts. For example, the name Apple may refer to more than 20 entities in Wikipedia, such as Apple Inc., Apple (band) and Apple Bank. Traditionally, there have been two distinct directions in EL to resolve the name ambiguity problem: one focusing on the effects of mention's context compatibility and the other dealing with the effects of document's topic coherence. EL methods based on context compatibility assume that "the referent entity of a mention is reflected by its context" (Mihalcea & Cosomai, 2007;Zhang et al., 2010;Zheng et al., 2010;Kataria et al., 2011;Sen 2012). For example, the context compatibility based methods will identify the referent entity of the mention Lion in Figure 1 is the entity Mac OS X Lion, since this entity is more compatible with its context words operating system and release than other candidates such as Lion(big cats) or Lion (band). EL methods based on topic coherence assume that "a mention's referent entity should be coherent with document's main topics" (Medelyan et al., 2008;Kulkarni et al., 2009;. For example, the topic coherence based methods will link the mention Apple in Figure 1 to the entity Apple Inc., since it is more coherent with the document's topic MAC OS X Lion Release than other referent candidates such as Apple (band) or Apple Bank.
In recent years, both of the above two EL directions have shown their effectiveness to some extent, and obviously they are complementary to each other. Therefore we believe that bring the above two directions together will enhance the EL performance. Traditionally, the above two directions are usually be brought together using a hybrid method (Zhang and Sim, 2011;Ratinov et al., 2011;, i.e., the context compatibility and the topic coherence are first separately modeled, then their EL evidence are combined through an additional model. For example, Zhang and Sim (2011) first models the context compatibility as a context similarity and the topic coherence as a similarity between the underlying topics of documents and KB entries, then these two similarities are combined through an additional SVM classifier for the final EL decision.
The main drawback of these hybrid methods, however, is that they model the context compatibility and the topic coherence separately, which makes it difficult to capture the mutual reinforcement effect between the above two directions. That is, the topic coherence and the context compatibility are highly correlated and their evidence can be used to reinforce each other in EL decisions. For example, in Figure 1, if the context compatibility gives a high likelihood the mention Apple refers to the entity Apple Inc., then this likelihood will give more evidence for this document's topic is about MAC OS X Lion, and it in turn will reinforce the topic coherence between the entity MAC OS X Lion and the document. In reverse, once we known the topic of this document is about MAC OS X Lion, the context compatibility between the mention Apple and the entity Apple Inc. can be improved as the importance of the context words operating system and release will be increased using the topic knowledge. In this way, we believe that modeling the above two directions jointly, rather than separately, will further improve the EL performance by capturing the mutual reinforcement effect between the context compatibility and the topic coherence.
In this paper, we propose a method to jointly model and exploit the context compatibility, the topic coherence and the correlation between them for better EL performance. Specifically, we propose a generative probabilistic model -called entity-topic model, which can uniformly model the text compatibility and the topic coherence as the statistical dependencies between the mentions, the words, the underlying entities and the underlying topics of a document by assuming that each document is generated according to the following two assumptions: 1) Topic coherence assumption: All entities in a document should be centered around the main topics of the document. For example, the entity Apple Inc. tends to occur in documents about IT, but the entity Apple Bank will more likely to occur in documents about bank or investment.
2) Context compatibility assumption: The context words of a mention should be centered on its referent entity. For example, the words computer, phone and music tends to occur in the context of the entity Apple Inc., meanwhile the words loan, invest and deposit will more likely to occur in the context of the entity Apple Bank.
In this way, the entity-topic model uniformly models the context compatibility, the topic coherence and the correlation between them as the dependencies between the observed information (the mentions and the words) in a document and the hidden information we want to know (the underlying topics and entities) through the global knowledge (including the topic knowledge, the entity name knowledge and the entity context knowledge). And the EL problem can now be decomposed into the following two inference tasks: 1) Predicting the underlying topics and the underlying entities of a document based on the observed information and the global knowledge. We call such a task the prediction task; 2) Estimating the global knowledge from data. Notice that the topic knowledge, the entity name knowledge and the entity context knowledge are all not previously given, thus we need to estimate them from data. We call such a task the knowledge discovery task. Because the accurate inference of the above two tasks is intractable in our entity-topic model, this paper also develops an approximate inference algorithm -the Gibbs sampling algorithm to solve them.
Contributions. The main contributions of this paper are summarized below: We propose a generative probabilistic model, the entity-topic model, which can jointly model and exploit the context compatibility, the topic coherence and the correlation between them for better EL performance; We develop a Gibbs sampling algorithm to solve the two inference tasks of our model: 1) Discovering the global knowledge from data; and 2) Collectively making accurate EL decisions. This paper is organized as follows. Section 2 describes the proposed entity-topic model. Section 3 demonstrates the Gibbs sampling algorithm. The experimental results are presented and discussed in Section 4. The related work is reviewed in Section 5. Finally we conclude this paper in Section 6.

The Entity-Topic Model for Entity Linking
In this section, we describe the proposed entitytopic model. In following we first demonstrate how to capture the context compatibility, the topic coherence and the correlation between them in the document generative process, then we incorporate the global knowledge generation into our model for knowledge estimation from data.

Document Generative Process
As shown in Section 1, we jointly model the context compatibility and the topic coherence as the statistical dependencies in the entity-topic model by assuming that all documents are generated in a topical coherent and context compatible way. In following we describe the document generative process. In our model, each document d is assumed composed of two types of information, i.e., the mentions and the words. Formally, we represent a document as: A document is a collection of M mentions and N words, denoted as d = {m 1 , …, m M ; w 1 , …, w N }, with m i the i th mention and w j the j th word.
For example, the document in Figure 1 is represented as d = {WWDC, Apple, Lion; at, the, conference, …}, where WWDC, Apple, Lion are the three mentions and the other are the words.
To generate a document, our model relies on three types of global knowledge, including: Topic Knowledge Á Á (The entity distribution of topics): In our model, all entities in a document are generated based on its underlying topics, with each topic is a group of semantically related entities. Statistically, we model each topic as a multinomial distribution of entities, with the probability indicating the likelihood an entity to be extracted from this topic. For example, we may have a topic Á Apple Inc: Á Apple Inc: = {Steve Jobs 0.12 , iPhone 0.07 , iPod 0.08 , …}, indicating the likelihood of the entity Steve Jobs be extracted from this topic is 0.12, etc.
Entity Name Knowledge Ã Ã (The name distribution of entities): In our model, all name mentions are generated using the name knowledge of its referent entity. Specifically, we model the name knowledge of an entity as a multinomial distribution of its names, with the probability indicating the likelihood this entity is mentioned by the name. For example, the name knowledge of the entity Apple Inc. may be Ã Apple Inc: Ã Apple Inc: = {Apple 0.51 , Apple Computer Inc. 0.10 , Apple Inc. 0.07 , …}, indicating that the entity Apple Inc. is mentioned by the name Apple with probability 0.51, etc.
Entity Context Knowledge » » (The context word distribution of entities): In our model, all context words of an entity's mention are generated using its context knowledge. Concretely, we model the context knowledge of an entity as a multinomial distribution of words, with the probability indicating the likelihood a word appearing in this entity's context. For example, we may have » Apple Inc: » Apple Inc: = {phone 0.07 , computer 0.10 , IT 0.06 , phone 0.002 , …}, indicating that the word computer appearing in the context of the entity Apple Inc. with probability 0.1, etc. Given the entity list E = {e 1 , e 2 , …, e E } in the knowledge base, the word list V = {w 1 , w 2 , …, w v }, the entity name list K = {n 1 , n 2 , …, n K } and the global knowledge described in above, the generation process of a document collection (corpus) D = {d 1 , d 2 , …, d D } is shown in Figure 2. To demonstrate the generation process, we also demonstrate how the document in Figure 1 can be generated using our model in following steps: Step 1: The model generates the topic distribution of the document as μ d μ d = {Apple Inc. 0.45 , Operating System(OS) 0.55 }; Step 2: For the three mentions in the document: i. According to the topic distribution μ d μ d , the model generates their topic assignments as z 1 =Apple Inc., z 2 = Apple Inc., z 3 = OS; ii. According to the topic knowledge Á Á Apple Inc. , Á Á OS and the topic assignments z 1 , z 2 , z 3 , the model generates their entity assignments as e 1 = Apple Worldwide Developers Conference, e 2 = Apple Inc., e 3 = Mac OS X Lion; iii. According to the name knowledge of the entities Apple Worldwide Developers Conference, Apple Inc. and Mac OS X Lion, our model generates the three mentions as m 1 =WWDC, m 2 = Apple, m 3 = Lion; Step 3: For all words in the document: i. According to the referent entity set in document e d = {Apple Worldwide Developers Conference, Apple Inc., Mac OS X Lion}, the model generates the target entity they describes as a 3 =Apple Worldwide Developers Conference and a 4 =Apple Inc.; ii. According to their target entity and the context knowledge of these entities, the model generates the context words in the document. For example, according to the context knowledge of the entities Apple Worldwide Developers Conference, the model generates its context word w 3 =conference, and according to the context knowledge of the entity Apple Inc., the model generates its context word w 4 = introduces.
Through the above generative process, we can see that all entities in a document are extracted from the document's underlying topics, ensuring the topic coherence; and all words in a document are extracted from the context word distributions of its referent entities, resulting in the context compatibility. Furthermore, the generation of topics, entities, mentions and words are highly correlated, thus our model can capture the correlation between the topic coherence and the context compatibility.

Global Knowledge Generative Process
The entity-topic model relies on three types of global knowledge (including the topic knowledge, the entity name knowledge and the entity context knowledge) to generate a document. Unfortunately, all three types of global knowledge are unknown and thus need to be estimated from data. In this paper we estimate the global knowledge through Bayesian inference by also incorporating the knowledge generation process into our model.
Specifically, given the topic number T, the entity number E, the name number K and the word number V, the entity-topic model generates the global knowledge as follows: For each topic z, our model samples its entity distribution Á z Á z from an E-dimensional Dirichlet distribution with hyperparameter ¯.

3) »j± » Dir(±) »j± » Dir(±)
Given the topic knowledge Á Á , the entity name knowledge Ã Ã and the entity context knowledge » »: 1. For each doc d in D, sample its topic distribution For each entity e, our model samples its context word distribution » e » e from a V-dimensional Dirichlet distribution with hyperparameter ± ±. Finally, the full entity-topic model is shown in Figure 3 using the plate representation.

The Probability of a Corpus
Using the entity-topic model, the probability of generating a corpus D={d 1 , d 2 , …, d D } given hyperparameters ® ®, ¯, °° and ± ± can be expressed as: where m d m d and e d e d correspondingly the set of mentions and their entity assignments in document d, w d w d and a d a d correspondingly the set of words and their entity assignments in document d.

Inference using Gibbs Sampling
In this section, we describe how to resolve the entity linking problem using the entity-topic model. Overall, there were two inference tasks for EL: 1) The prediction task. Given a document d, predicting its entity assignments (e d e d for mentions and a d a d for words) and topic assignments ( z d z d ). Notice that here the EL decisions are just the prediction of per-mention entity assignments (e d e d ).
2) The knowledge discovery task. Given a corpus D={d 1 , d 2 , …, d D }, estimating the global knowledge (including the entity distribution of topics Á Á, the name distribution Ã Ã and the context word distribution » » of entities) from data.
Unfortunately, due to the heaven correlation between topics, entities, mentions and words (the correlation is also demonstrated in Eq. (2.1), where the integral is intractable due to the coupling between μ μ, Á Á, Ã Ã and » » ), the accurate inference of the above two tasks is intractable. For this reason, we propose an approximate inference algorithmthe Gibbs sampling algorithm for the entity-topic model by extending the well-known Gibbs sampling algorithm for LDA (Griffiths & Steyvers, 2004). In Gibbs sampling, we first construct the posterior distribution P (z; e; ajD) P (z; e; ajD) , then this posterior distribution is used to: 1) estimate μ μ, Á Á, Ã Ã and » »; and 2) predict the entities and the topics of all documents in D. Specifically, we first derive the joint posterior distribution from Eq. (2.1) as: P (z; e; ajD) / P (z)P (ejz)P (mje)P (aje)P (wja) P (z; e; ajD) / P (z)P (ejz)P (mje)P (aje)P (wja) where is the probability of the joint topic assignment z to all mentions m in corpus D, and is the conditional probability of the joint entity assignments e e to all mentions m in corpus D given all topic assignments z z, and is the conditional probability of all mentions m m given all per-mention entity assignments e e, and is the conditional probability of the joint entity assignments a a to all words w in corpus D given all per-mention entity assignments e e, and is the conditional probability of all words w w given all per-word entity assignments a a . In all above formulas, ¡(:) ¡(:) is the Gamma function, C DT dt C DT dt is the times topic t has been assigned for all mentions in document d, have similar explanation. Based on the above joint probability, we construct a Markov chain that converges to the posterior distribution P (z; e; ajD) P (z; e; ajD) and then draw samples from this Markov chain for inference. For entity-topic model, each state in the Markov chain is an assignment (including topic assignment to a mention, entity assignment to a mention and entity assignment to a word). In Gibbs sampling, all assignments are sequentially sampled conditioned on all the current other assignments. So here we only need to derive the following three fully conditional assignment distributions: 1) P (z i = tjz ¡i ; e; a; D) P (z i = tjz ¡i ; e; a; D): the topic assignment distribution to a mention given the current other topic assignments z ¡i z ¡i , the current entity assignments e e and a a; 2) P (e i = ejz; e ¡i ; a; D) P (e i = ejz; e ¡i ; a; D) : the entity assignment distribution to a mention given the current entity assignments of all other mentions e ¡i e ¡i , the current topic assignments z z and the current entity assignments of context words a a; 3) P (a i = ejz; e; a ¡i ; D) P (a i = ejz; e; a ¡i ; D) : the entity assignment distribution to a context word given the current entity assignments of all other context words a ¡i a ¡i , the current topic assignments z z and the current entity assignments e e of mentions. Using the Formula 3.1-3.5, we can derive the above three conditional distributions as (where m i is contained in doc d): where the topic assignment to a mention is determined by the probability this topic appearing in doc d (the 1 st term) and the probability the referent entity appearing in this topic (the 2 nd term); P (e i = ejz; e ¡i ; a; D) / where the entity assignment to a mention is determined by the probability this entity extracted from the assigned topic (the 1 st term), the probability this entity is referred by the name m (the 2 nd term) and the contextual words describing this entity in doc d (the 3 rd term); where the entity assignment to a word is determined by the number of times this entity has been assigned to mentions in doc d (the 1 st term) and the probability the word appearing in the context of this entity (the 2 nd term). Finally, using the above three conditional distributions, we iteratively update all assignments of corpus D until coverage, then the global knowledge is estimated using the final assignments, and the final entity assignments are used as the referents of their corresponding mentions.
Inference on Unseen Documents. When unseen documents are given, we predict its entities and topics using the incremental Gibbs sampling algorithm described in (Kataria et al., 2011), i.e., we iteratively update the entity assignments and the topic assignments of an unseen document as the same as the above inference process, but with the previously learned global knowledge fixed.
Hyperparameter setting. One still problem here is the setting of the hyperparameters ® ®, ¯, °° and ± ±. For ® ® and ¯, this paper empirically set the value of them to ® = 50=T ® = 50=T and ¯= 0:1 = 0:1 as in Griffiths & Steyvers(2004). For °°, we notice that K°K° is the number of pseudo names added to each entity, when °= 0°= 0 our model only mentions an entity using its previously used names. Observed that an entity typically has a fixed set of names, we set °° to a small value by setting K°= 1:0 K°= 1:0. For ± ±, we notice that V ± V ± is the number of pseudo words added to each entity, playing the role of smoothing its context word distribution. As there is typically a relatively loose correlation between an entity and its context words, we set ± ± to a relatively large value by fixing the total smoothing words added to each entity, a typical value is V ± V ± = 2000.

Experiments
In this section, we evaluate our method and compare it with the traditional EL methods. We first explain the experimental settings in Section 4.1-4.4, then discuss the results in Section 4.5.

Knowledge Base
In our experiments, we use the Jan. 30, 2010 English version of Wikipedia as the knowledge base, which contains over 3 million entities. Notice that we also take the general concepts in Wikipedia (such as Apple, Video, Computer, etc.) as entities, so the entity in this paper may not strictly follow its definition.

Evaluation Criteria
This paper adopted the same performance metrics used in the Kulkarni et al. (2009), which includes Recall, Precision and F1. Let M * be the golden standard set of the EL results (each EL result is a pair (m, e), with m the mention and e its referent entity), M be the set of EL results outputted by an EL system, then these metrics are computed as: where two EL results are considered equal if and only if both their mentions and referent entities are equal. As the same as Kulkarni et al.(2009), 3 http://www.cse.iitb.ac.in/~soumen/doc/QCQ/ Precision and Recall are averaged across documents and overall F1 is used as the primary performance metric by computing from average Precision and Recall.

Baselines
We compare our method with five baselines which are described as follows: Wikify!. This is a context compatibility based EL method using vector space model (Mihalcea & Csomai, 2007). Wikify! computes the context compatibility using the word overlap between the mention's context and the entity's Wikipedia entry.
EM-Model. This is a statistical context compatibility based EL method described in , which computes the compatibility by integrating the evidence from the entity popularity, the entity name knowledge and the context word distribution of entities.
M&W. This is a relational topic coherence based EL method described in . M&W measures an entity's topic coherence to a document as its average semantic relatedness to the unambiguous entities in the document.
CSAW. This is an EL method which combines context compatibility and topic coherence using a hybrid method (Kulkarni et al., 2009), where context compatibility and topic coherence are first separated modeled as context similarity and the sum of all pair-wise semantic relatedness between the entities in the document, then the entities which can maximize the weighted sum of the context compatibility and the topic coherence are identified as the referent entities of the document.
EL-Graph. This is a graph based hybrid EL method described in , which first models the context compatibility as text similarity and the topic coherence of an entity as its node importance in a referent graph which captures all mention-entity and entity-entity relations in a document, then a random walk algorithm is used to collectively find all referent entities of a document.
Except for CSAW and EL-Graph, all other baselines are designed only to link the salient name mentions (i.e., key phrases) in a document. In our experiment, in order to compare the EL performances on also the non-salient name mentions, we push these systems' recall by reducing their respective importance thresholds of linked mentions.

Overall Performance
We compared our method with all the above five baselines. For our method, we estimate the global knowledge using all the articles in the Jan. 30, 2010 English version of Wikipedia, and totally there were 3,083,158 articles. For each article, the mentions within it are detected using the methods described in Medelyan et al.(2008) and all terms in an article are used as context words, so a term may both be a mention and a context word. The topic number of our model is T = 300 T = 300 (will be empirically set in Sect 4.5.2). To train the entitytopic model, we run 500 500 iterations of our Gibbs sampling algorithm to converge. The training time of our model is nearly one week on our server using 20 GB RAM and one core of 3.2 GHz CPU. Since the training can be done offline, we believe that the training time is not critical to the realworld usage as the online inference on new document is very quick. Using the above settings, the overall results are shown in Table 1. 1) By jointly modeling and exploiting the context compatibility and the topic coherence, our method can achieve competitive performance: ○ 1 compared with the context compatibility baselines Wikify! and EM-Model, our method correspondingly gets 43% and 19% F1 improvement; ○ 2 compared with the topic coherence baselines M&W, our method achieves 28% F1 improvement; ③ compared with the hybrid baselines CSAW and EL-Graph, our method correspondingly achieves 11% and 7% F1 improvement.
2) Compared with the context compatibility only and the topic coherence only methods, the main advantage of our method is that, rather than only achieved high entity linking precision on salient mentions, it can also effectively link the non-salient mentions in a document: this is demonstrated in our method's significant Recall improvement: a 32~52% Recall improvement over baselines Wikify!, EM-Model and M&W. We believe this is because a document usually contains little evidence for EL decisions on non-salient mentions, so with either only context compatibility or only topic coherence the evidence is not enough for EL decisions on these non-salient mentions, and bring these two directions together is critical for the accurate EL on these mentions.
3) Compared with the hybrid methods, the main advantage of our method is the improvement of EL precision (a 11~16% improvement over baselines CSAW and EL-Graph), we believe this is because: ○ 1 Our method can further capture the mutual reinforcement effect between the context compatibility and the topic coherence; ○ 2 The traditional hybrid methods usually determine the topic coherence of an entity to a document using all entities in the document, in comparison our method uses only the entities in the same topic, we believe this is more reasonable for EL decisions.

Parameter Tuning
One still parameter of our method is the topic number T. An appropriate T will distribute entities into well-organized topics, in turn it will capture the co-occurrence information of entities. Figure 4 plots the F1 at different T values. We can see that the F1 is not very sensitive to the topic number and with T = 300 T = 300 our method achieves its best F1 performance.

Detailed Analysis
In this section we analyze why and how our method works well in detail. Generally, we believe the main advantages of our method are: 1) The effects of topic knowledge. One main advantage of our model is that the topic knowledge can provide a document-specific entity prior for EL. Concretely, using the topic knowledge and the topic distribution of documents, the prior for an entity appearing in a document d is highly related to the document's topics: This prior is obviously more reasonable than the "information less prior" (i.e., all entities have equal prior) or "a global entity popularity prior" . To demonstrate,   Table 3. The 3 topics where the fruit Apple has the largest P(e|z) 2) The effects of a fine-tuned context model. The second advantage of our model is that it provides a statistical framework for fine-tuning the context model from data. To demonstrate such an effect, Table 4 compares the EL performance of ① the entity-topic model with no context model is used (No Context), i.e., we determine the referent entity of a mention by deleting the 3rd term of the formula P (e i = ejz; e ¡i ; a; D) P (e i = ejz; e ¡i ; a; D) in Section 3; ② with the context model estimated using the entity's Wikipedia page (Article Content), ③ with the context model estimated using the 50 word window of all its mentions in Wikipedia (Mention Context) and; ④ with the context model in the original entity-topic model (Entity-Topic Model). From Table 4 we can see that a fine-tuned context model will result in a 2~7% F1 improvement. 3) The effects of joint model. The third advantage of our model is that it jointly model the context compatibility and the topic coherence, which bring two benefits: ① the mutual reinforcement between the two directions can be captured in our model; ② the context compatibility and the topic coherence are uniformly modeled and jointly estimated, which makes the model more accurate for EL.

EL Accuracies on TAC 2009 dataset
We also compare our method with the top 5 EL systems in TAC 2009 and the two state-of-the-art systems (EM-Model and EL-Graph) on TAC 2009 data set in Figure 5 (For EL-Graph and our method, a NIL threshold is used to detect whether the referent entity is contained in the knowledge base, if the knowledge base not contains the referent entity, we assign the mention to a NIL entity). From Figure 5, we can see that our method is competitive: 1) Our method can achieve a 3.4% accuracy improvement over the best system in TAC 2009; 2) Our method, EM-Model and EL-Graph get very close accuracies (0.854, 0.86 and 0.838 correspondingly), we believe this is because: ○ 1 The mentions to be linked in TAC data set are mostly salient mentions; ○ 2 The influence of the NIL referent entity problem, i.e., the referent entity is not contained in the given knowledge base: Most referent entities (67.5%) on TAC 2009 are NIL entity and our method has no special handling on this problem, rather than other methods such as the EM-Model, which affects the overall performance of our method.

Related Work
In this section, we briefly review the related work of EL. Traditionally, the context compatibility based methods link a mention to the entity which has the largest compatibility with it. Cucerzan (2007) modeled the compatibility as the cosine similarity between the vector space representation of mention's context and of entity's Wikipedia entry. Mihalcea & Csomai (2007), Bunescu & Pasca (2006), Fader et al. (2009, Gottipati et al.(2011) and Zhang et al.(2011) extended the vector space model with more information such as the entity category and the acronym expansion, etc.  proposed a generative model which computes the compatibility using the evidences from entity's popularity, name distribution and context word distribution. Kataria et al.(2011) and Sen (2012) (2011) employed the ranking techniques to further take relations between candidate entities into account.
On the other side, the topic coherence based methods link a mention to the entity which are most coherent to the document containing it. Medelyan et al. (2008) measured the topic coherence of an entity to a document as the weighted average of its relatedness to the unambiguous entities in the document. Milne and Witten (2008) extended Medelyan et al. (2008)'s coherence by incorporating commonness and context quality. Bhattacharya and Getoor (2006) modeled the topic coherence as the likelihood an entity is generated from the latent topics of a document. Sen (2012) modeled the topic coherence as the groups of co-occurring entities. Kulkarni et al. (2009) modeled the topic coherence as the sum of all pair-wise relatedness between the referent entities of a document. Hoffart et al.(2011) modeled the topic coherence of an entity as its node importance in a graph which captures all mention-entity and entity-entity relations in a document.

Conclusions and Future Work
This paper proposes a generative model, the entitytopic model, for entity linking. By uniformly modeling context compatibility, topic coherence and the correlation between them as statistical dependencies, our model provides an effective way to jointly exploit them for better EL performance.
In this paper, the entity-topic model can only link mentions to the previously given entities in a knowledge base. For future work, we want to overcome this limit by incorporating an entity discovery ability into our model, so that it can also discover and learn the knowledge of previously unseen entities from a corpus for linking name mentions to these entities.