{'fraction_non_alphanumeric': 0.03579709627938501, 'fraction_numerical': 0.011004887097349552, 'mean_word_length': 4.540217391304348, 'pattern_counts': {'":': 0, '<': 10, '<?xml version=': 0, '>': 10, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 9, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Topological quantum phase transition in Kane-Mele-Kondo lattice model

We systematically explore the ground-state phase diagram of the Kane-Mele-Kondo lattice model on the honeycomb lattice, in particular, we focus on its magnetic properties which has not been studied in the previous publication[Feng, Dai, Chung, and Si, Phys. Rev. Lett. \textbf{111}, 016402 (2013)]. Beside the Kondo insulator found in that paper, two kinds of antiferromagnetic spin-density-wave phases are identified. One is the normal antiferromagnetic spin-density-wave state and the other is a nontrivial topological antiferromagnetic spin-density-wave state with a quantized spin Hall conductance and a helical edge-state. The quantum spin Hall insulator is found to be absent since it is always unstable to antiferromagnetic spin-density-wave states at least at the mean-field level in our model. Furthermore, the transition between the two spin-density-wave phases are topological quantum phase transition described by the three-dimensional quantum electrodynamics, in which conduction electrons contribute to the low-energy Dirac fermions while the spin-wave fluctuation of local spins gives rise to an effective dynamic U(1) gauge-field. Such nontrivial transition shows radical critical thermodynamic, transport and single-particle behaviors, which provide a fingerprint for this transition. Additionally, the transition of antiferromagnetic spin-density-wave states to the Kondo insulator is found to be first-order. The introduction of two novel magnetic phases and their topological quantum phase transition show rich and intrinsic physics involving in the Kane-Mele-Kondo lattice model.


I. INTRODUCTION
The interplay of the strong electron correlation and the band topology attracts much attention in recent years after the discovery of the two dimensional quantum spin Hall insulator (QSH) and three-dimensional topological band insulator. [1][2][3][4][5][6][7] Both of these two novel insulators are the bulk band insulator but have nontrivial gapless edge or surface states protected by the time-reversal symmetry. 7 When strongly electron correlated effect is considered, understanding the fate of the bulk topology and the corresponding boundary states is still a challenge in the condensed matter community.  Despite of the hardness of this issue, many new and exotic states of matter have been uncovered, e.g., the topological Mott insulator, 8 fractional Chern/topological insulator, [9][10][11][12][13][14][15][16] Z 2 -fractionalized Chern/topological insulator, [17][18][19] topological Kondo insulator [20][21][22] and topological antiferromagnetic spin-density-wave phase. [23][24][25][26][27][28] Among them, the concept of topological Kondo insulator (TKI) has been proposed as an extension of the original topological (band) insulators into the heavy fermion systems. 20 In usual topological insulators the spin-orbit coupling is encoded in a spin-dependent hopping amplitudes between different unit cells. The TKI is produced by the spin-orbit coupling associated with the hybridization between conduction and local electrons and its resulting energy band is also heavily renormalized by the local strong correlation of local electrons. [20][21][22] In comparison to the conventional Kondo insulator (KI), 38-40 the TKI remains a renormalized band insulator but it pos-sesses certain conducting surface-states, which could be identified experimentally from the usual KI compounds. Theoretically, the three-dimensional TKI may be relevant to certain heavy fermion compounds with strong spin-orbit coupling. Very recent experimental studies on the classic Kondo insulator SmB 6 seem to support that SmB 6 is a putative three-dimensional topological Kondo insulator. [41][42][43] This motivates us further study of the TKI in these strongly correlated systems.
Very recently, a microscopic Kondo lattice model on the two-dimensional honeycomb lattice with spin-orbit coupling between conduction electrons has been proposed to understand the interplay between spin-orbit coupling (bulk topology) and Kondo screening effect in twodimensional system. 35 In that paper the authors found the KI and the QSH insulators paramagnetic phases, and identified continuous second-order quantum phase transition between these two phases.
However, it should be noted that the magnetic longrange ordered states, which commonly appear and rather relevant for both the theoretical Kondo lattice-like models and realistic heavy fermion compounds, have not been clearly explored in the previous work of Feng et al. 35 . As a matter fact, in some heavy fermion materials, the antiferromagnetism is even near by or coexistent with unconventional superconductivity. In this case the magnetic fluctuation is believed to be driving force of such superconducting instability. 44 Furthermore, many experimentally observed exotic non-Fermi liquid behaviors are also close to the magnetic instability and its related quantum critical points. 45 On the other hand, intuitively, based on our previous studies on the kondo lattice models on the honeycomb lattice, 27,46 we suspect that the paramagnetic states are not stable to magnetic phases, particularly, the QSH insulator found by Feng et al. is always unstable to the antiferromagnetic spin-wave-density (SDW) states if the Kondo coupling between local spins and conduction electrons is not vanishing small. Another interesting feature of including the magnetic order is that it could give rise to an example of the mentioned topological antiferromagnetic SDW (T-SDW) phase. [25][26][27][28] The T-SDW phase is identified as an antiferromagnetic SDW state with some nontrivial bulk topological quantum numbers or bulk charge/spin response. This state is indeed beyond the classic Landau's symmetry-breaking classification of matter since the T-SDW and the normal SDW (N-SDW) states have identical bulk symmetry (breaking the spinrotation invariance due to the antiferromagnetic order). 47 In some sense, the T-SDW states can be considered as an interesting extension of the usual topological insulator with the broken time-reversal symmetry by the magnetic long-range order.
In the present work, we discuss these interesting issues in the Kane-Mele-Kondo lattice model proposed in Ref. 35 at half-filling. To be specific, we will use the mean-field decoupling to obtain magnetic T and N-SDW phases as well as the paramagnetic KI and QSH insulator phases. Importantly, the magnetic ordered T and N-SDW states are novel features of the current work beyond that in the previous paper of Feng et al. 35 . We compare ground-state energies of those phases to establish the global phase diagram for the Kane-Mele-Kondo lattice model. After identifying those phases, the corresponding topological Chern number is explicitly calculated and the Chern-Simons and quantum electrodynamics-like effective field theories are utilized to capture the possible topological properties by taking important quantum fluctuations around the mean-field solutions into account.
It is found that the low energy effective theory of the Z 2 topological ordered T-SDW phase can be constructed by the hydrodynamics approach, which is successfully applied to the well-known integer and fractional quantum Hall effect. 47 The hydrodynamics approach gives rise to a mutual U(1)× U(1) Chern-Simons action coupled to the external electromagnetic and spin gaugefield. Furthermore, the bulk-boundary correspondence of the Chern-Simons theory provides a candidate of the helical edge-state for the T-SDW phase. More interestingly, the topological quantum phase transition between the T and N-SDW phases is described by the unexpected three-dimensional quantum electrodynamics, which appears in many important condensed theory issues, e.g. the gauge theory description of hightemperature supconductivity, 49 the quantum spin liquids in quantum magnets 50 and the deconfined quantum criticality 51 . Such emergent quantum electrodynamics is a novel feature of the present work and is also not covered in Ref. 35. In the present case, we find that conduction electrons contribute to the low-energy Dirac fermions while the spin-wave fluctuation gives rise to an effective dynamic U(1) gauge-field. Critical quasiparticle, thermodynamic and transport behaviors of the emergent quantum electrodynamics are analyzed at the lowest-order and the one-loop levels.
Furthermore, we find that if the Kondo coupling is turned on, the QSH insulator becomes unstable in comparison to the antiferromagnetic SDW states, thus at least within framework of our mean-field theory, the ground-state phase diagram will not cover the QSH state found in Ref. 35 and only T/N-SDW and KI phases remain. The transition of these two magnetic states to the KI phase is first-order. Additionally, we have also discussed the doped Kane-Mele-Kondo lattice model and ferromagnetism may be relevant when the system is deviated from the half-filling case. The relation to the TKI and many implications from both numerical and experimental studies are also argued with an aim toward the realization of the Kane-Mele-Kondo model and confirmation of the various phases of the ground-state phase diagram.
To briefly summarize, we focus on the magnetic properties of the Kane-Mele-Kondo lattice model, and find two kinds of novel ordered phases, namely, the N-and T-SDW phases. Furthermore, the topological quantum phase transition between two magnetic phases found is a novel feature for this model, and its effective field theory is the celebrated 2+1D QED, which appears in many important and interesting issues, for example, deconfined quantum criticality, gauge-theory description of the high temperature superconductivity and quantum spin liquids in quantum magnets. Finally, we also systematically discussed the topological properties of the model including topological Chern number, effective Chern-Simons fieldtheory for the Z 2 topological ordered phase, the edgestate Hamiltonian, and so on. These results have not clearly identified in the previous work published by Feng et al. 35 The remainder of this paper is organized as follows. In Sec. II, Kane-Mele-Kondo lattice model is introduced and its mean-field Hamiltonian is derived. In Sec. III, we present detailed study of antiferromagnetic ordered phases. The T and N-SDW states are uncovered and their physical properties are inspected from the single particle band to topological properties. In Sec. IV, fluctuation correction is considered for the antiferromagnetic phases and effective field theories are derived and analyzed. We construct the ground-state phase diagram of Kane-Mele-Kondo lattice model in Sec.V. Sec. VI and VII give some important discussions and extensions. Finally, Sec. VIII is devoted to a brief conclusion. filling, where H K is the well-known Kane-Mele Hamiltonian, 3 which supports the quantum spin Hall (QSH) effect with helical edge states (Right and left movers are locked with the up and down spin, respectively), t and t ′ are the nearest-neighbor and the next-nearest-neighbor hopping, respectively. 7 The phase ϕ ij = ± 1 2 π is introduced to give rise to a quantum spin Hall effect without external magnetic fields (the so-called QSH) and the positive phase is gained with anticlockwise hopping. Besides, the pseudofermion representation for local spins has been utilized as H denotes the magnetic instability due to the polarization of conduction electrons by local spins while H ⊥ describes the local Kondo screening effect resulting from spin-flip scattering process of conduction electrons by local moments.

A. Some background
The interplay between the Kondo screening effect and the magnetic instability on the honeycomb lattice without the nontrivial next-nearest-neighbor hopping term (t ′ ) has been studied by some of the present authors in the previous work. 46 In that work, we find either a direct first-order transition or a possible coexistence of the Kondo insulator and the N-SDW state by using the extended mean-field decoupling. Furthermore, we have also discussed the Haldane-Kondo lattice model where the conduction electrons are described by the spinful Haldane model. 27 We found an extra antiferromagnetic phase, namely the T-SDW state, which has a quantized charge Hall conductance coexistent the magnetic order. The transition of T-SDW state into the conventional N-SDW state is found to fall into a XY-like topological quantum phase transition.
The model we study here is different to the Haldane-Kondo lattice model and more richer physics has been found in comparison to the previous one. In particular, the T-SDW in the present paper is different to the one in Ref. 27 since the current T-SDW phase has a quantized spin Hall conductance and the edge-state is the helical edge-state. We now use the hydrodynamics approach to establish an effective theory for T-SDW phase by extracting the bulk topological properties and to identify existence of the non-chiral edge-state. More interest-ingly, the phase transition between T and N-SDW is now described by the three-dimensional quantum electrodynamics (QED-3) in contrast to the XY transition in the previous work. The QED-3 theory may give rise to a nontrivial universal conductance, which is also different to the previous work. In addition, we also provide many discussion on the doped system, which is important but not considered before.
Recently, Feng et al. have studied the above model Eq. 1 by using the slave boson mean-field theory. They mainly focus on the possible paramagnetic states, i.e., the Kondo insulator and the quantum spin hall insulator (two dimensional topological insulator). They found a secondorder transition between those two paramagnetic states. However, the magnetic ordered states, which commonly appear in the Kondo lattice-like models and realistic heavy fermion compounds, are not studied. Intuitively, based on our previous studies on the kondo lattice models on the honeycomb lattice, we suspect that the paramagnetic states are not stable to magnetic phases, particularly, the quantum spin Hall insulator is always unstable to the antiferromagnetic SDW states if the Kondo coupling is not vanishing small. In the present paper we will discuss this issue in the following sections and we will explore the various transitions among those magnetic and paramagnetic phases and establish the global phase diagram for the ground-state of the Kane-Mele-Kondo lattice model.

B. Remark on the T-SDW state
In the present paper, the T-SDW state is an antiferromagnetic long-range ordered state with the quantized spin Hall effect and a helical edge-state. Similar T-SDW phase seems to be found in a numerical way and the time-reversal symmetry may be recovered in the edge, thus the helical edge-state is protected by the recovered symmetry. 28 In contrast, in Ref. 25 and 27, the T-SDW phase represents a time-reversal symmetry-breaking antiferromagnetic state with the quantized charge Hall effect. The broken time-reversal symmetry leads to two chiral edge-states in the latter T-SDW phase, which is similar to the case of usual electric or bosonic integer quantum Hall effect where the external/effecitve large magnetic field explicitly breaks the time-reversal invariance and leads to energy bands with nontrivial Chern number. 47,52 C. Mean-field Hamiltonian of Kane-Mele-Kondo lattice model In this subsection, we present the mean-field decoupling and the consultant mean-field Hamiltonian, which is our starting point to study the ground-state phase diagram of the Kane-Mele-Kondo lattice model [Eq.
(1)]. Although the mean-field treatment of generic two-dimensional lattice models needs caution in strong correlation scenario, it may provide qualitatively correct physics if quantum fluctuations are weak. In the following discussion, we assume that the mean-field treatment is able to capture the basic features of the Kane-Mele-Kondo lattice model. Further study on this model in terms of advanced numerical techniques is desirable.
By using the mean-field decoupling as presented in Refs.27 and 53 for the longitudinal and transverse interaction term H , H ⊥ , respectively, the resultant meanfield Hamiltonian reads where N s is the number of lattice sites. Several mean-field parameters have been defined as The mean-field parameters m d , m c correspond to magnetization of local spins and conduction electrons, respectively, and serve as usual magnetic order parameters. The nonvanishing Kondo hybridizing parameter V denotes the onset of Kondo screening effect and it can also be considered as an order parameter although the formation of Kondo screening does not imply any global symmetrybreaking. Besides, since we are considering a half-filled lattice, the local constraint of the pseudofermion has been safely neglected at the present mean-field level with chemical potential set to zero. 53 With the mean-field Hamiltonian Eq. (2) in hand, we can extract some but not all possible physical phases. Firstly, when all mean-field parameters are zero, the Hamiltonian will only have the conduction electron part and this just corresponds to the quantum spin Hall state. 3 Secondly, when Kondo hybridizing parameter V is non-vanishing but m c = m d = 0, this is the Kondo state. Since the system is half-filling, one may expect that it is the Kondo insulating state. 39 Thirdly, if we have m c , m d = 0 with V = 0, the corresponding state is the antiferromagnetic ordered state. Finally, if all of the mean-field parameters are nonzero, one may find a state where the antiferromagnetic long-range order coexists with the Kondo screening. 46,53 Interestingly, we will find an antiferromagnetic ordered state with a quantized spin Hall conductance, which is beyond our naive inspect due to blindness for the subtle topological properties. Detailed study on these possible states is the purpose of the present paper but the coexistent state will be left for future study due to its complexity and subtlety.
Before moving to the discussion on the possible states of Kane-Mele-Kondo lattice model in the next section, it is helpful to give a brief argument on the basic properties of Kane-Mele model and its corresponding Chern-Simons treatment since these issues may not appear in the literature of heavy fermions and the same technique will be used in the next section. Another motivation to do this is that when the mean-field order parameters are all zero, the Hamiltonian Eq. (2) is reduced to the pure Kane-Mele model, which is just the quantum spin Hall insulator found in Ref. 35.

D. The Kane-Mele model
In this subsection, we give a brief discussion about the basic property of the Kane-Mele model 3 It is useful to rewrite this single-particle Hamiltonian in the momentum space as where we have defined f (k) = e −ikx + 2e ikx/2 cos( 2 k y ) and A, B representing two nonequivalent sublattices of the honeycomb lattice, respectively. Then, by diagonalizing the above Hamiltonian, one obtains the quasiparticle energy band as which preserves the particle-hole symmetry and also the spin degeneracy. It is well-known that for 3 √ 3t ′ < t, the excitation gap mainly opens near six Dirac points (Only two of them are nonequivalent in fact). 29 Then, expanding both f (k) and γ(k) near two nonequivalent Dirac points ± K = ±(0, 4π 3 √ 3 ), respectively, the gap can be found as ∆ gap = 6 √ 3t ′ and the quasiparticle energy . The most interesting property of the Kane-Mele model is that it gives rise to a vanished charge Hall conductance but a nonzero quantized spin Hall conductance in contrast to the usual quantum Hall or quantum anomalous Hall effect. 54 The corresponding quantized spin Hall conductance can be readily calculated by 55 For the Kane-Mele model, the single particle Hamiltonian can be rewritten asĥ kσ = d σ (k) ·σ with d σ = (−tRef (k), tImf (k), 2t ′ σγ(k)),d σ = d σ /|d σ | and σ being the usual Pauli matrices. Then, by inserting the expression ofd σ into the formula of C σ 1 , one obtains C ↑ 1 = −C ↓ 1 = 1 and σ SH = e h while the charge Hall con- In literature, the ground-state of the Kane-Mele model is often called the quantum spin Hall insulator (topological insulator) since it exhibits a quantized spin Hall conductance, which has the gapped bulk spectrum with a gapless helical edgestate protected by the time-reversal symmetry. 3

E. The low-energy effective theory for the Kane-Mele model
For the discussion of the low-energy physics, an effective 2 + 1D massive Dirac action can be obtained by expanding original Kane-Mele model around two nonequivalent Dirac points ± K, Here the same indices mean summation. We introduce the effective mass m = −3 √ 3t ′ of Dirac fermions and set the effective Fermi velocity v F = 3 2 t to unit. The Dirac fields are defined as ψ 1σ = (c 1Aσ , c 1Bσ ) T , ψ 2σ = (c 2Aσ , −c 2Bσ ) T andψ aσ = ψ † aσ γ 0 with a = 1, 2 denoting the states near the two nonequivalent Dirac points ± K = ±(0, 4π 3 √

) and
T implying the transposition manipulation. We should remind the reader that the spin-up and spin-down fields ψ aσ acquire opposite mass term as can be seen from the above effective action, which leads to vanished charge Hall conductance but a nonzero quantized spin Hall conductance.
Having obtained the effective massive Dirac action in 2+1D, it is interesting to see its physical response to the external electromagnetic field representing the usual scalar and vector potentials, respectively. The electromagnetic field can be readily introduced into the Dirac action by the conventional minimal coupling, namely, The charge e is setting to unit for simplicity.) However, one will find that the Chern-Simons action for the electromagnetic field A c µ vanishes due to the opposite mass for different spin-flavors, which means the charge Hall conductance is zero. (This is consistent with the results of the previous subsection.) But, one can add the artificial external spin gauge-field A s µ to see the nontrivial spin response. 56 Thus, the resulting effective Dirac action coupled with the external electromagnetic and spin fields reads By integrating out the Dirac fields, we get an effective mutual Chern-Simons action, which represents the electromagnetic response of the massive Dirac fermions to the external electromagnetic field A c µ and artificial external spin gauge-field A s µ (for details, one can refer to Appendix A), where the pre-factor 2 comes from the nonequivalent Dirac points, ǫ µνλ is the usual all-antisymmetric tensor and we have dropped out the regular Maxwell term (∼ F 2 µν ) since the low energy physics is dominated by the Chern-Simons term alone. We should emphasize that although the effective Chern-Simons action is used here, it does not imply any fractionalization or nontrivial topological order (A characteristic signature of the topological order is the ground-state degeneracy depending on the topology of the system.) because no emergent gauge fields or fractionalized quasiparticles exist in the present case. 47 Since the effective mass m = −3 √ 3t ′ < 0, the above Chern-Simons action can be simplified to the following form Now, one can see that the quantized spin Hall conduc- If we reintroduce h = 2π and charge e, this spin hall conductance reads σ SH = e h , which reproduces the correct result in terms of spin Chern number in the previous subsection. We note that the existence of the spin Hall conductance relies on the existence of the charge degree of freedom since the spin gauge-field A s µ couples to the electromagnetic field A c µ in the Chern-Simons action. Otherwise, if the elementary particles (here the electrons) do not carry charge, their spin Hall conductance is obviously zero. The case here is rather different to the spin quantum Hall effect, 57,58 whose spin Hall conductance does not rely on the charge degree of freedom.

III. THE ANTIFERROMAGNETIC SPIN-DENSITY-WAVE STATE
In this section, we will present the full discussion of the antiferromagnetic spin-density-wave state including the mean-field equations, the effective action and the edgestates.
A. The antiferromagnetic spin-density-wave state For the case with J ≫ J ⊥ , in general, one expects that the antiferromagnetic SDW state to be the stable ground-state of Kondo lattice model on the honeycomb lattice due to its bipartite feature. 39 To study the possible antiferromagnetic ordered state, we assume that no Kondo screening exists (V = 0) while magnetic order parameters m c , m d are not vanishing in the mean-field Hamiltonian Eq. (2). Then, the resultant Hamiltonian reads

Possibility of a SDW state with a quantized spin Hall conductance
Here we see that the conduction electrons c and local pseudofermions d are clearly decoupled at the present mean-field level. The nonzero magnetic order parameters m c , m d mean that the state obtained by such a hamiltonian is magnetically ordered. Besides, we may note that the conduction electron part H c is similar to the Kane-Mele model Eq. (3) with an extra effective coupling from the magnetic order of local spins. Based on such an observation, if the conduction electron part is able to support a quantized spin Hall conductance, the obtained antiferromagnetic state should be identified as the topological one. Using the same method in the discussion of Kane-Mele model in the previous section, the spin Chern numbers are calculated as where we assume t ′ , m d , J > 0 and the step function Θ(x) is defined by Θ(x) = 1 (= 0) for x > 0 (x < 0). Therefore, if C ↑ 1 , C ↓ 1 are nonzero, we will have an antiferromagnetic state with a quantized spin Hall conductance . To find such interesting possibility, we have to solve the above mean-field Hamiltonian below (Note that m d has to be solved by the mean-field equations).

The mean-field equations of the SDW states
Diagonalizing the mean-field Hamiltonian, we can easily derive ground-state energy of the antiferromagnetic SDW state per site as and two self-consistent equations from minimizing E AF M g with respect to magnetization m d and m c , respectively.
For further analytical treatment, one may use a simplified linear density of state (DOS) ρ(ε) = |ε|/Λ 2 when transforming the summation over momentum k into integral on energy ε with Λ ≃ 2.33t being high-energy cutoff. Thus, t|f (k)| is replaced by |ε| to simplify corresponding calculations and γ(k) is replaced by ±3 √ 3t ′ near two nonequivalent Dirac points ± K = ±(0, 4π 3 √ 3 ). From these two equations, one obtains m d = 1/2 and while the ground-state energy per site for the antiferromagnetic SDW state reads From above equations, we observe that the local spins are fully polarized (m d = 1/2) while the conduction electrons have small magnetization. It is also noted that when t ′ = 0 (no next-nearest-neighbor hopping), the above m c correctly recovers the value in our previous work. 46 Meanwhile, the low-lying quasiparticle excitations in the antiferromagnetic SDW state have the energy E ±±σ (k) = ± (3tk/2) 2 + (J /4 ± 3 √ 3t ′ ) 2 and E ±±σ (k) = ±J m c /2. The former corresponds to the conduction electrons while the latter, which is nondispersive, relates to the local spins/pseudofermions. It should be noted that the gap around the Dirac points only closes when the condition J /4 = 3 √ 3t ′ is fully satisfied. Otherwise, any low-lying quasiparticle excitations in the antiferromagnetic SDW state are clearly gapped. Thus, we may conclude that the antiferromagnetic SDW state we obtained is mainly an insulating state (except for the case with J /4 = 3 √ 3t ′ ) with fully polarized local spins (m d = 1/2) while conduction electrons only partially polarize (m c < 1/2). This feature is similar to the previous study on square lattice, thus confirms the validity of our current treatment. 53 Furthermore, since we have calculated the order parameter m d = 1/2, according to the discussion in the previous paragraph, the spin Chern numbers are found as . This case corresponds to the vanished gap for quasiparticle excitations and can be identified as a topological quantum phase transition between the N-SDW state and the T-SDW one. More details will be pursued in the next subsection.

B. The physics of the T-SDW state
Having established the existence of the T-SDW and N-SDW states, we will use effective theories to get deeper insight into those states and their corresponding topological quantum phase transition in this and next two subsections.

The T-SDW state
Performing the same treatment as for the Kane-Mele model in Sec.II on the mean-field Hamiltonian, we obtains the following effective action for the antiferromagnetic SDW state where the sign function sgn(x) = 1 for x > 0 and sgn(x) = −1 for x < 0 and we have used the fact m 1↑ < 0, m 1↓ > 0. It is easy to see that a quantized spin Hall conductance with the value σ SH = e 2 /h is obtained if −m = 3 √ 3t ′ > J /4 (See the second subsection of Sec.II for the calculation of the quantized spin Hall conductance.). In contrast, when 3 √ 3t ′ is smaller than J /4, no such quantized spin Hall conductance can be found and the corresponding effective Chern-Simon term vanishes.
Therefore, it seems that even in the antiferromagnetic SDW state, there exists a quantized spin Hall conductance if the quasiparticle gap is still dominated by the next-nearest neighbor hopping (3 √ 3t ′ ) rather than the antiferromagnetic order (J /4). Thus, we have uncovered a topological antiferromagnetic SDW state, namely, the T-SDW state with a quantum spin Hall effect for 3 √ 3t ′ > J /4.

The hydrodynamics approach and the topological property of the T-SDW state
It is also interesting to see the topological properties in the T-SDW state. Since the T-SDW state has nonzero spin Chern numbers or quantized spin Hall conductance, follow the hydrodynamics approach for integer/fractional quantum Hall (IQH/FQH) states, 47 we first write down the the conservation current for each spin-flavor J 1µ = 1 2π ǫ µνλ ∂ ν a 1λ , J 2µ = 1 2π ǫ µνλ ∂ ν a 1λ (here 1 for spin-up and 2 for spin-down electrons) Due to the fact that the spin-up (down) electrons has spin Chern number C ↑ 1 = 1 (C ↓ 1 = −1), we can attribute Chern-Simons terms to each flavor as −i 4π ǫ µνλ a 1µ ∂ ν a 1λ for spin-up and +i 4π ǫ µνλ a 2µ ∂ ν a 2λ for spin down ones. Next, the conserved currents are coupled to the external electromagnetic field A c µ with identical charge (iA c µ (J 1µ − J 2µ )) and coupled to the spin gauge-field A s µ with charge 1/2 (spin-up electrons) and −1/2 (spin-down electrons) ( i 2 A s µ (J 1µ − J 2µ )). Combining these terms, we have the following effective abelian Chern-Simons action where we have defined the so-called K matrix as K 11 = 1, K 22 = −1, K 12 = K 21 = 0 for I, J = 1, 2 and the corresponding charge and spin vectors read q = (1, 1) T and s = (1, −1) T , respectively. 47 A care reader may note that we have introduced the spin gauge-field A s µ and the spin vector s I , which do not appear in usual Chern-Simons actions for description of topological states. For the physical observable, the quantized charge Hall conductance is calculated as σ H = ν e 2 h = 0 with ν = q T K −1 q = 0 and the quantized spin Hall conductance reads σ SH = ν s e 2h = e h with ν s = s T K −1 q = 2, thus the effective action correctly reproduces the quantized spin Hall conductance and vanished charge Hall conductance obtained in the previous subsection.
Additionally, some readers may wonder whether there exists a nontrivial topological order since the effective theory Eq. (11) is a Chern-Simons action. This issue can be seen as follows. Firstly, one finds Det[K] = 1, which means no state with topological order and fractional excitations are involved in the present system 59 . In contrast, systems supporting fractionalized excitations should require a K-matrix with |Det[K]| > 1 since the groundstate degeneracy on a torus is equal to Det[K]. 52 ). Secondly, if one examines the statistics of the elementary quasiparticle (the electron) in terms of the so-call exchange statistical angle, without any surprise, one will find such angle is π which implies that the exchange of two identical quasiparticle gives rise to a π phase in their wavefunction. Therefore, we conclude that the elementary quasiparticle is the usual electron as expected.

The edge-state of the T-SDW state
It is well-known that many topological states, e.g. QSH, IQH and FQH insulators, have nontrivial edgestates in contrast to trivial band insulators. 7,47 Therefore, we will discuss property of the edge-state for the T-SDW state.
Since the topological features are described in terms of the Chern-Simons action Eq. (11), according to the standard bulk-edge correspondence for the effective abelian Chern-Simons theory, the gapless edge states is described by two counter-moving bosonic modes 47 with c denoting the non-universal velocity of edge states and φ I being the bosonic representation for the two edgestate modes (I = 1 for the spin-up mode while I = 2 for the spin-down mode). Because the right (left) mover carries spin-up (spin-down) degree of freedom, this edgestate is just the helical edge-state which is widely studied in the field of the quantum spin Hall effect/twodimensional topological insulator. 7 To see this correspondence more clearly, one can refermionize above action by introducing the fermion operator ψ I ∝ e iφI and the resulting action reads And the corresponding Hamiltonian formalism is simple which is just the usual helical edge-state in QSH insulator. 7 [The chiral bosonic field φ I is now rewritten .] The basic properties of the topological antiferromagnetic SDW (T-SDW) state is summarized in Fig. 1 Firstly, if the conservation of particle number is preserved, in general, the local interacting terms will be the following form 60 Obviously, the dangerous umklapp term H 3 vanishes due to ψ I ψ I = ψ † I ψ † I = 0(I = 1, 2). H 1 , H 2 do not vanish but they cannot gap out the edge states since they only correspond to the usual forward scattering.
Secondly, besides these interacting terms, the mass term H m = M dx[ψ † 1 ψ 2 + ψ † 2 ψ 1 ], which can result from weak impurity scattering, is important. 60 Adding this backscattering term to the free Hamiltonian part Eq. (13), one can see an excitation gap ∆ = 2|M |. Thus, if no symmetry is imposed, this backscattering term will destroy the gaplessness of the edge-state of T-SDW state described by Eq. (13).
We remind the reader that in the field of QSH insulator, one assumes the time-reversal symmetry is still preserved at the edge, so such impurity scattering term is prohibited since it is not invariant under the time-reversal transformation (ψ 1 → ψ 2 , ψ 2 → −ψ 1 , i → −i). In our case, the bulk has the antiferromagnetic SDW order and this magnetic order inevitably breaks the time-reversal symmetry in the bulk. As for the edge, we suspect that the time-reversal invariance is in fact persevered in the T-SDW state although it is broken in the bulk. This conjecture is motivated by the numerical study of the Bernevig-Hughes-Zhang model with Hubbard-U term. 28 In their case, the T-SDW state is also found with nontrivial quantized spin Hall conductance, whose value is found to approach unit ( e h ). Based on this observation, those authors deduce that the antiferromagnetic order is suppressed at the edges, and the helical edge states are topologically protected against magnetic instability, in spite of the existence of the bulk antiferromagnetic order. 28 Therefore, we will assume that the gaplessness of helical edge-state Eq. (13) of T-SDW state is stable to impurity as time-reversal symmetry is conserved at the edge.
Since the spin-up and spin-down part of the gapless edge-state contributes e 2h to the spin Hall conductance, 54 one obtains that the total spin Hall conductance is quantized to e h , as expected. Obviously, this robust quantized value for the spin Hall conductance is protected by the time-reversal symmetry as we have assumed. However, we should emphasize that the nonzero quantized spin Chern number does not necessarily imply the existence of the gapless edge-state, as shown in Ref. 61, where the authors found that the spin Chern number is well quantized but the edge-states are gapped when the timereversal symmetry of the Kane-Mele model is broken by the Rashba spin-orbit coupling or the magnetic exchange field. Therefore, one should be careful when naively using the bulk-edge correspondence.

C. The N-SDW state
When J /4 > 3 √ 3t ′ , the Chern-Simons term vanishes and this indicates that this case is just the usual antiferromagnetic SDW state, namely, the N-SDW. If we examine the T-SDW and N-SDW from the point of view of symmetry, we find that these two SDW states have identical physical symmetry with broken spin-rotation invariance. However, from the discussion of previous sections, the spin response of them is rather different (The former has quantized spin Hall conductance while the latter one does not.) which can be considered as a useful criteria to distinguish those two states. In this sense, the T-SDW state provides a simple example beyond the symmetrybreaking based classification of states of matter.
where we have integrated out massive ψ 1↑ and ψ 1↓ . Clearly, this quantum phase transition point is described by two gapless free Dirac fields and its thermaldynamic and transport properties are familiar. According to the conventional scaling theory, 62 the thermaldynamic properties are described by the free energy f (T ) ∼ T d+z z where T, d, z denoting temperature, spatial dimension and dynamical critical exponent. For our present system, d = 2 and z = 1, which reflects the emergent Lorentz symmetry (Time and space have the same scaling under the standard scaling transformation.). Therefore, the specific heat is found as The magnetization has the similar behavior as the specific heat M (T ) ∼ T 2 due to the same scaling of the external magnetic field and the temperature. Thus, the magnetic susceptibility reads χ(T ) ∼ ∂M ∂B | B→T ∼ T . For the electromagnetic response, we can integrate out ψ 2↑ and ψ 2↓ to get the following action where the polarization function Π µν (q) = e 2 16 (δ µν − qµqν q 2 ) q 2 is calculated in Appendix B. Therefore, the charge conductance is found as e 2 when σ xy (ω) = σ yx (ω) = 0. (Note that the charge conductance is independent of the frequency.) As comparison, we note that the charge conductance are all vanished in both T and N-SDW states.
More importantly, when considering fluctuation effect in next section, we will find that the mentioned topological quantum phase transition is in fact described by the three dimensional quantum electrodynamics, namely the so-called QED-3, where Dirac fermions are coupled to the dynamic U(1) gauge-field. It is noted that this interesting feature is in contrast to our previous study reported in Ref. 27. The main findings in this subsection are systematically summarized in Fig. 2. √ 3t ′ = J /4 with Λ ≃ 2.33t being highenergy cutoff. As noted, there exists a nonzero charge conductance σ = π 4 e 2 at the TQPT point.

IV. FLUCTUATION EFFECT BEYOND THE MEAN-FIELD TREATMENT FOR ANTIFERROMAGNETIC SPIN-DENSITY WAVE STATES
A. fluctuation effect in antiferromagnetic spin-density wave states Here, we discuss the spin fluctuation effect in the antiferromagnetic SDW states, which is neglected in the previous mean-field treatment in Sec. II.
Firstly, let us recall the effective Dirac action [Eq. (10)]  25, when the antiferromagnetic order is well established, the effect of spin fluctuations is equivalent to introduce an effective U(1) dynamic gauge-field b µ into the above action. In fact, the dynamic U(1) gaugefield b µ represents the usual spin-wave modes (the Goldstone mode of the antiferromagnetic SDW background) since it has a linear spectrum if we integrate out the Dirac fermions and set external fields A c µ = A s µ = 0. This point will be clearly seen in discussion below.
It is noted that different spin flavor of electrons have the opposite gauge charge of b µ , which just indicates that the gauge-field b µ describes the spin fluctuations of ordered magnetic background. After integrating out Dirac fermions, one can obtain For N-SDW state, we have m 1↑ , m 2↓ < 0 and m 1↓ , m 2↑ > 0, thus the resulting action vanishes. Oppositely, if we consider the T-SDW states, (m 1↑ , m 2↑ < 0 and m 1↓ , m 2↓ > 0) the effective action reads Here the Maxwell term for b µ is reintroduced. Now, if we just turn off the external field A c µ , we get only the Maxwell term for b µ , which clearly denotes a linear spectrum. Since b µ is the dynamical field, one must integrate it out to obtain the physical classical action. Integrating b µ out, one has the following action where the external electromagnetic field A c µ acquires a mass term m 2 c ∝ g, which represents the quantum fluctuation (spin-wave) correction in the T-SDW state. We should emphasize that although there exists an extra mass term for the external electromagnetic field, the spin Hall conductance is still unchanged since it depends the response to the spin gauge-field A s µ .
) Therefore, we may conclude that the spin fluctuation in the antiferromagnetic SDW states does not lead to any qualitative changes on the mean-field results on the previous section.
Some careful readers may wonder that since the external electromagnetic field A c µ acquires a mass term, does it indicate certain condensation of bosonic objects? We think that the since the antiferromagnetic SDW states are just the result of the condensation of electrons in the particle-hole channel and the spin-wave or the gaugefield b µ is the Goldstone mode of such condensation, the mentioned mass term can be considered as a result of the Anderson-Higgs mechanism, which means the electromagnetic field A c µ kills the gapless Goldstone mode and acquires a mass term. In this subsection, we discuss the spin fluctuation effect at the topological quantum phase transition point between the N-and the T-SDW states (m = −3 √ 3t ′ = −J /4) in Sec. II. Here, the effective mass of Dirac action is m 1↑ = −m 1↓ = −J /2 < 0 and m 2↑ = m 2↓ = 0. Thus, ψ 1↑ , ψ 2↓ are massive and can be safely integrated out while one should not integrate out ψ 2↑ , ψ 2↓ due to gaplessness of them. Then, following the treatment of last subsection, one obtains where we have integrated out two massive modes, which leads to a mutual Chern-Simons term for A c µ , A s µ and b µ . If one is not interested in the external field response of A c µ and A s µ , we may neglect them and get the effective action as Interestingly, this action is just the well-known three dimensional quantum electrodynamics (QED-3) but with opposite coupling for different flavor Dirac fermions. The QED-3 action appears in the gauge-field description of high temperature superconductivity and some quantum magnetism problems. 49 But, in those gauge-field description, the Dirac fermions are not the physical electrons so the Green's function of the Dirac fermions are not physical observable while in our case, the Dirac fermions are real electrons, thus it indeed describes true physics. In other words, if we obtain the Green's function of the Dirac fermions in our model, one can get its corresponding local density of state, which could be measured by local differential conductance in scanning tunneling microscopy experiments.
Unfortunately, for the QED-3, we do not know the exact results for the gauge-field and fermion Green's functions. And here, we only present the one-loop/large-N result. 63 Integrating out Dirac fermions at the one-loop level, we can get the action for the gauge-field b µ , So the Green's function for the gauge-field is D µν . Then, using this Green's function, one can compute the self-energy correction for the Dirac fermions, is the Green's function for free Dirac fermions. Calculating this integral (for details, one can refer to Appendix C), we find Σ(k) ≃ 16i 3π 2 γ µ k µ ln( Λ |k| ) and the Dirac fermion Green's function reads Superficially, this indicates a rather large anomalous dimension η ≃ 0.54 for Dirac fermions, but the one-loop result is not reliable and the realistic situation is not clear at present stage. Since we do not have a good understanding of the QED-3 theory, the present topological quantum phase transition point is obviously poorly understood and this interesting issuer is left for future study.
One may wonder that since the gauge-field b µ does not have its own Chern-Simons term, the action of the QED-3 may be meaningless due to the well-known confinement. 64,65 However, we should note that the gauge-field b µ is in fact coupled to the physical external electromagnetic field A c µ and since the A c µ is obviously deconfined in the three-dimensional time-space, the gaugefield b µ has to be deconfined in the present case. In other words, deconfinement of b µ is protected by its coupling to the physical external electromagnetic field. Similar case also appears in the study of the quantum phase transition of bosonic integer quantum Hall phases. 66,67 After all, the topological quantum phase transition between the T and N-SDW states can be described by an effective QED-3 theory though understanding exactly physics of this theory is beyond our ability.

V. THE KONDO INSULATING STATE, QUANTUM SPIN HALL INSULATOR AND KONDO BREAKDOWN
Having discussed the properties of magnetic ordered states in the previous sections, it is time to study paramagnetic states of the Kane-Mele-Kondo lattice model in order to compare with the previous results. The paramagnetic states include the Kondo insulator and the quantum spin Hall insulator, which are also discussed in Ref. 35. So, we will give a brief discussion on these two states based on our mean-field Hamiltonian Eq. (2). The basic physical features are consistent with the previous work. 35

A. The Kondo insulating state
Another interesting case appears when J ≪ J ⊥ . It is natural to expect that a Kondo insulating state arises in this situation for half-filling. 39,53 Following the same methology of treating antiferromagnetic SDW state, we can get the mean-field Hamintonian and its ground-state energy per site for the expected Kondo insulating state with V = 0 but no magnetic orders m d = m c = 0 And the four quasi-particle energy bands are Minimizing E Kondo g with respect to the Kondo hybridization parameter V , we can obtain is in contrast to usual mean-field result β = 1/2, and this is due to the low-energy linear DOS of conduction electrons near Dirac points of the honeycomb lattice.
For this single-particle Hamiltonian, its ground-state energy is readily to calculate as E 0 3 ] which comes solely from free conduction electrons. Comparing E 0 g and E Kondo g , one clearly recovers the critical coupling J c ⊥ , which justifies the above simple picture.
Since the Kondo insulating state is unstable to the QSH state when J ⊥ < J c ⊥ , one may wonder whether the QSH state appears between the Kondo insulating state and the antiferromagnetic SDW states. Particularly, if the QSH state is stable, there will exist a critical point corresponds to the Kondo breakdown 45,68-71 on the honeycomb lattice when J ⊥ = J c ⊥ , which has been studied in Refs.35 and 72. Actually, the authors in Ref.72 also discusses the U(1) gauge fluctuation around the above mean-field ground-state and found that the critical theory for this putative Kondo breakdown point is described by multi-flavor Dirac fermions and an O(2) boson field coupled to a dynamic U(1) gauge field. The details can be consulted that reference.
However, recalling of the ground-state energy of the antiferromagnetic SDW state E AF M g , we find that it is KI is the Kondo insulator and QSHI denotes the quantum spin Hall insulator. The quantum phase transition between the KI and QSHI is second order and is considered as a Kondo breakdown transition. However, when magnetic orders is introduced, QSHI is unstable and this QPT is meaningless. always lower than E 0 g for any positive Kondo coupling J and the next-nearest-neighbor hopping t ′ . Therefore, we conclude that the nontrivial quantum spin Hall state in intermediate coupling seems unfavorable based on our present mean-field treatment and the stability of QSH state in Ref. 35 may require extra frustration interactions or certain geometrical frustrations. 73,74 (For example, a J 1 − J 2 Heisenberg exchange term for local spins may be added into the Kondo lattice model and if appropriately tuning the ratio of J 1 /J 2 may frustrate the magnetic ordered states and lead to the wanted paramagnetic QSH phase. 50 ) In other words, we only expect the system supports the phases of SDW states, the paramagnetic Kondo insulator state and the phase transitions between them.

C. Fluctuation in Kondo insulator and QSH insulator
Since both the Kondo insulator and QSH state are gapped, the small fluctuation cannot change the basic physics of those states. To be specific, for the Kondo insulator, the fluctuation will mainly come from the fluctu- we treat V as a dynamic complex bosonic field. However, since V acquires nonzero expectation in the Kondo insulator phase, the possible gapless fluctuation of it is its phase degree of freedom. However, such phase degree of freedom can be absorbed into the gapped Lagrangian multiplier field. 75 So, V cannot give rise to any dangerous gapless fluctuation and thus the Kondo insulator is stable.
As for the QSH insulator, unless the fluctuation due to the Kondo hybridizing and magnetic order terms is comparable to the single-particle gap of the QSH state, we still expect the QSH state to be stable to fluctuations. In the antiferromagnetic SDW regime, N-SDW denotes the normal antiferromagnetic SDW state while the T-SDW represents the topological antiferromagnetic SDW state with a quantum spin Hall conductance and helical edge-states. The boundary of these two kinds of SDW states is the topological quantum phase transition, which is determined by condition of 3 √ 3t ′ = J/4 with Λ ≃ 2.33t being high-energy cutoff.

VI. CONSTRUCTING THE GROUND-STATE PHASE DIAGRAM OF KANE-MELE-KONDO LATTICE
Based on the previous sections, in which the T-SDW, N-SDW, QSH insulator and Kondo insulator states are individually discussed, in Fig. 4 we show the global ground-state phase diagram. Here we do not consider possible coexistence regime for simplicity. In this phase diagram, the QSH insulator is absent due to its energy is always higher than that of antiferromagnetic SDW states, as discussed in last section. The Kondo insulator is the same as that in Ref. 35. In addition, the two kinds of antiferromagnetic SDW states are central results of the present work and the transition between them is topological quantum phase transition determined by the condition of close of gap 3 √ 3t ′ = J/4. The phase boundary between two antiferromagnetic SDW states and the Kondo insulator is found to be first-order. 62

VII. FURTHER DISCUSSIONS AND POSSIBLE EXTENSIONS
A. Doping case: Possibility of a nontrivial topological phase The above Sections have focused on the half-filled Kane-Mele-Kondo lattice model since in this case, one can expect nontrivial topological properties when the spin Chern number is nonzero due to full occupation of certain single-particle energy bands. However, if the sys- tem is doped, it is difficult to expect nontrivial topological state like QSH insulator or the T-SDW state studied in the present paper. However, one notes that the authors in Ref. 35 found that the 1/4 and 3/4 filling cases also have nontrivial Z 2 topological bulk invariant and their surface-state is a new helical edge-state, which contains the contribution from both conduction and local electrons (spins). (Recalling that the usual helical edgestate only includes the contribution of conduction electrons and this appears in the QSH and T-SDW states, which we have studied in the main text.) To get more insight into the mentioned situation of the 1/4 and 3/4 filling, we plot the quasi-particle energy bands in Fig.5. We can see that when the 1/4 filling is considered, the lowest band is filled and any singleparticle excitation to the other bands has a gap. The 3/4 filling case is the same to 1/4 filling. However, we cannot use the formula C In usual Kondo lattice model, when the conduction electron has small filling fraction, the ferromagnetism may be relevant and could occupy some regime of the ground-state phase diagram. 39,76 This means that at small filling, the antiferromagnetic states are unstable to ferromagnetic states. This may indicate that when one dopes the Kane-Mele-Kondo lattice strongly deviating the half-filling case, one could find a ferromagnetic tems may realize our proposed model on the honeycomb lattice. [77][78][79] F. Three-dimensional system with interplay of magnetism and topology Till now, the theoretical study of the topological Kondo states is only focused on the topological Kondo insulator though we know that even the most promising topological Kondo insulator material SmB 6 can also show an antiferromagnetic state under external pressure. Thus, it is interesting to theoretically discuss the transition from the putative topological Kondo insulator to the antiferromagnetic ordered state in certain threedimensional lattice models. Recalling that the topological Kondo insulator is described by the Anderson lattice models in Refs.20 and 22, introducing a Heisenberg exchange interaction between the spin degree of freedom of local spins may include the effect of magnetism into the original models.

VIII. CONCLUSION
In summary, we have obtained the ground-state phase diagram of the Kane-Mele-Kondo lattice model on the honeycomb lattice at half-filling. It is found that the paramagnetic Kondo insulator and the magnetic normal and topological antiferromagnetic spin-density-wave states occupy the regimes of the phase diagram while the quantum spin Hall insulator is excluded. The topological magnetic state has the same symmetry as the normal one but it has an extra quantized spin Hall conductance and a nontrivial helical edge-state. Interestingly, those topological properties can be captured in terms of a mutual Chern-Simons theory and the stability of the helical edge-state is protected by recovered time-reversal symmetry. Furthermore, there exists a novel topological quantum phase transition between the two kinds of magnetic phases, whose critical behaviors may be described by a three-dimensional quantum electrodynamics. In addition, a first-order quantum phase transition is found between the Kondo insulating state and the antiferromagnetic states. We hope the present work may be helpful for further studies on the interplay between conduction electrons and the densely localized spins on the honeycomb lattice.

ACKNOWLEDGMENTS
The work was supported partly by NSFC, PCSIRT (Grant No. IRT1251), the Program for NCET, the Fundamental Research Funds for the Central Universities and the national program for basic research of China.

Appendix A: Derivation of Chern-Simons action
Here, we would like to give a brief derivation of the effective Chern-Simons action Eq. (6) from the Dirac action Eq. (5) but without the spin gauge-field A s µ . Introducing the spin gauge-field is easy to implement as just a replacement (A c µ → A c µ + σ 2e A s µ ). Firstly, we consider the following Dirac action Then, integrating out Dirac fermions one obtains where Π µν (q) = −e 2 8π|m| ǫ µνλ q λ is calculated at one-loop level. We have also used the identity Tr(γ µ γ ν γ λ ) = 2iǫ µνλ with γ 0 = τ z , γ 1 = τ x and γ 2 = τ y while ≃ 1 8π|m| for |q| ≪ |m|. Therefore, the effective Chern-Simon action is obtained as Consider Eq. (5), we have four-flavor Dirac fields and the effective action reads Since m 1σ = m 2σ = mσ, one gets The above action is just Eq. (6) if e is set to unit.