{'fraction_non_alphanumeric': 0.037487310273107685, 'fraction_numerical': 0.011290761878822393, 'mean_word_length': 4.434337997847147, 'pattern_counts': {'":': 0, '<': 14, '<?xml version=': 0, '>': 6, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 7, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
The quantum ground state of self-organized atomic crystals in optical resonators

Cold atoms, driven by a laser and simultaneously coupled to the quantum field of an optical resonator, can self-organize in periodic structures. These structures are supported by the optical lattice, which emerges from the laser light they scatter into the cavity mode, and form when the laser intensity exceeds a threshold value. We study theoretically the quantum ground state of these structures above the pump threshold of self-organization, by mapping the atomic dynamics of the self-organized crystal to a Bose-Hubbard model. We find that the quantum ground state of the self-organized structure can be the one of a Mott-insulator or a superfluid, depending on the pump strength of the driving laser. For very large pump strengths, where the intracavity intensity is maximum and one would expect a Mott-insulator state, we find intervals of parameters where the system is superfluid. These states could be realized in existing experimental setups.


I. INTRODUCTION
Self-organization in systems of atoms and light has been observed in numerous experiments, some of which are reported in Refs. [1][2][3][4][5][6]. Several experimental realizations show the formation of spatially-ordered atomic structures, which organize in the potential they form [1,5,6]. The basic mechanism behind the observed dynamics can be summarized by considering that the refractive index of the atomic medium, which is related to the atomic spatial density, is itself determined by the light fields via the mechanical effects of atom-photon interactions. Hence, the effective dynamics the atomic center of mass undergoes is determined by potentials and/or forces, which in return depend on the atom position and velocity distributions [7].
In this context, one remarkable example is the formation of regular patterns of atoms in the standing wave of a high-finesse optical cavity, arising when the atoms are transversally driven by lasers. Experimental signatures are the phase-locking between the field at the cavity output and the driving laser phase, according to two possible values with difference equal to π. This occurs when the laser frequency is smaller than the cavity-mode frequency and its intensity exceeds a threshold value [5]. The phenomenon, first predicted by numerical simulations according to a semiclassical model for atoms and light [8], can be understood in terms of the atoms being trapped in the potential which originates from the light coherently scattered from the laser into the cavity mode. The largest intracavity amplitude is supported by the atomic configurations in which all atoms scatter in phase into the resonator, corresponding to the patterns where the interatomic distance is a multiple of the wavelength of the cavity field. Since the possible configurations which fulfill this condition in a standing-wave cavity are two, one shifted with respect to the other by half a wavelength, the difference between the possible phases of the emitted field is exactly π. A further theoretical study discussed the phenomenon in terms of second-order phase transition and determined the pump threshold within a semiclassical, mean-field model [9].
Additional novel phenomena arise in this system when the quantum mechanical properties of light and matter are relevant. Theoretical works studied the dynamics of entanglement between atoms and fields during selforganization in a similar setup [10][11][12]. In Ref. [13] the pump threshold for self-organization at T = 0 was determined when the atoms are assumed to be forming a Bose-Einstein condensate. Most recently, the onset of selforganization has been experimentally observed in a Bose-Einstein condensate coupled to an optical resonator [6]. In such situation, an open question regards the nature of the quantum state of the system when the pump intensity is increased well above the self-organization threshold value. In this regime, in fact, the height of the potential increases, and one would expect localization of the atoms at the potential minima when the atom number is increased and hence the field in the cavity is larger. On the other hand, interparticle interactions due to s-wave collisions compete with such dynamics, favoring situations in which the number of atoms per site is small. This framework is reminiscent of the paradigmatic Mottinsulator / superfluid quantum phase transition [14], and in particular its realization with ultracold atoms in optical lattices [15]. In the case here considered, however, the dependence of the potential on the atomic distribution (and in particular, the expectation that its depth increases with the atomic density) makes it a priori unclear whether incompressible states can exist at large laser intensities.
In this article we study the quantum ground state of this self-organized system, in the setup sketched in Fig. 1. In particular, we determine under which conditions incompressible states exist at large pump intensities. For this purpose, we extend a procedure, developed for an atom-cavity system in which the cavity mode is pumped by a laser [16][17][18], and derive a Bose-Hubbard model for the atoms trapped in the potential their generate, whose coefficients depend on the atomic density. Using the strong-coupling expansion [19], we identify the parameter region where incompressible states of the selforganized atomic pattern may exist. This study finds particular motivation from experimental progress, which achieved the coupling of ultracold atoms with the optical mode of high-finesse cavities [6,[20][21][22][23][24][25]. Indeed, we will argue that incompressible self-organized states could be observed in the setups of existing experiments [6,26]. Ω κ λ FIG. 1: (color online) Setup of the system, showing matterwave density distribution (blue region) in the effective potential created inside an optical resonator by coherent scattering of laser photons (red line). Here, Ω is the Rabi frequency, giving the strength of the coupling between the atoms and the laser field, λ is the wavelength of laser and cavity mode, and κ is the rate at which photons leak from the cavity mode. The quantum state of the system could be inferred by measuring the light at the cavity output, or by Bragg spectroscopy using a weak probe (see Sec. V).
This article is organized as follows. The equations at the basis of the physical model are introduced in Sec. II, and the relevant approximations leading to the derivation of the effective atomic dynamics in the light potential are extensively discussed. In Sec. III, we assume that the atoms are tightly bound at the minima of the cavity potential inside the cavity, and derive an effective Bose-Hubbard model, whose coefficients are analytically determined within a modified Gaussian ansatz. The parameter regimes, where incompressible states of the selforganized atomic gas are found, are derived in Sec. IV within the strong coupling expansion method [19]. Conclusions and outlook are presented in Sec. V, while the appendix provides further details complementing the discussion in Secs. III and IV.

II. THE MODEL
Identical bosonic atoms, with mass m, are confined inside an optical resonator, which is assumed to be a highfinesse cavity sustaining well resolved modes. The atoms are prepared at ultralow temperature T and interact with each other by means of s-wave scattering. Furthermore, the atoms couple to light via a dipolar transition between the electronic ground and excited state |g and |e at the optical frequency ω 0 . In particular, the atomic dipole is simultaneously driven by a laser and by the field of one cavity mode, both linearly polarized and whose wave vectors are perpendicular to one another, according to the geometry of the setup sketched in Fig. 1. We assume the atoms to be strongly confined in the plane perpendicular to the cavity axis, here chosen to be the x-axis, so that the motion is essentially along x. In absence of the resonator, the atomic dynamics for the atoms in states |g and |e are governed by the Hamiltonian H 0 + H eg , where with g j the strength of the state-dependent collisional interaction, and where H eg describes s-wave scattering between atoms in different atomic states. The Hamiltonian in Eq. (1) is written in second quantization, where operators Ψ i (x) and Ψ † i (x) annihilate and create an atom at the position x and in the electronic state i = g, e. The atomic field operators obey the bosonic The cavity is assumed to be a high-finesse resonator sustaining well-resolved modes, of which the mode at frequency ω c is (quasi) resonant with the atomic transition. We denote by a and a † the annihilation and creation operators of a cavity photon at frequency ω c . The coherent coupling between atoms and light is described by a Hamiltonian in the Rotating-Wave approximation, which reads and which is here reported in the reference frame rotating at the laser frequency ω p . Here, Ω(x) denotes the strength of the coupling to the laser field (Rabi frequency), g 0 is the cavity-mode vacuum Rabi frequency, cos(kx) gives the spatial dependence of the cavity-mode function with wave vector k, while ∆ a = ω p − ω 0 and ∆ c = ω p − ω c denote the detuning of the laser frequency from the atom and from the cavity-mode frequencies, respectively. The coherent dynamics of coupled atoms and electromagnetic field are then governed by the full Hamiltonian This far we just considered Hamiltonian dynamics. We now introduce the sources of noise and dissipation, which in the present model are spontaneous radiative decay of the the electronic excited state |e at the lifetime γ and cavity losses at rate 2κ due to the finite transmittivity of the mirrors. Noise is here described within the Heisenberg-Langevin formalism, and the Heisenberg-Langevin equations for the atomic and field operators read [18] Here, the noise operators f in (t) and a in (t) are the Langevin forces, with vanishing mean value and , see for instance Ref. [27].

A. Effective dynamics
We now assume a time-scale separation, such that the electronic variables relax to a local steady state on a much shorter time scale than the one characterizing the dynamics of external and cavity degrees of freedom. This regime for the cavity degrees of freedom requires that the parameters satisfy |∆ a + iγ/2| ≫ |Ω|, g 0 √ N , |∆ c |, κ. Moreover, a change in the external degrees of freedom can be neglected over the typical time scale of the internal degrees of freedom provided that |∆ a + iγ/2| ≫ κ B T /h, with κ B Boltzmann's constant. In addition, we assume |∆ a | ≫ |Ω(x)|, g 0 √ N ≫ γ/2, so that we can identify a time scale in which coherent evolution is taking place while dissipation due to spontaneous decay can be discarded. In these limits Eq. (4) can be set to zero and at leading order gives such that Eq. (5) takes the forṁ where U 0 = g 2 0 /∆ a is the maximal depth of the singlephoton potential and S 0 = g 0 Ω/∆ a is the maximal amplitude of scattering a laser photon into the cavity mode by a single atom. In writing Eq. (7) we have assumed, moreover, that Ω(x) = Ωf (x), with |f (x)| ≤ 1. The spatial distribution of atoms and fields is now contained in the operators Y and Z, which read and act on the Hilbert space of the atoms. The operators Y and Z are Hermitian and commute, [Y, Z] = 0, as they both depend solely on the atomic density. They are the quantum analogous of the semiclassical "bunching parameter" B = j cos 2 (kx j )/N and of the "spatial order parameter" Θ = j cos(kx j )/N , respectively, which are defined for an ensemble of N atoms at the positions x 1 , . . . , x N , see for example [9]. These parameters characterize a semiclassical mean-field description of selforganization of the atoms in the cavity field, such that for Θ → ±1 the atoms are in a self-organized state at the even or odd antinodes of the cavity standing wave, while B provides their degree of localization about these points [28].
The cavity field can be eliminated from the matterwave equations assuming a second characteristic time scale determined by the rate |∆ c + iκ|, in which the cavity field approaches a local steady state. This requires that the effective coupling strength between matter waves and cavity field is smaller than this rate, namely |∆ c + iκ| ≫ |S 0 | √ N , |U 0 |N , and that the uncoupled matter wave dynamics is slower, |∆ c + iκ| ≫ κ B T /h. In this limit, one finds a ∼ a (0) , with where we have used that the operators in the numerator commute with the operators in the denominator. Using in Eq. (3) the solutions for Ψ e (x) and a, Eq. (6) and Eq. (10), at lowest order in perturbation theory we finḋ Here a (0) is now a function of the atomic density and hence does not commute with the field operator Ψ g (x).

B. Mechanical effects of the cavity field
Equation (10) for the cavity-field operator a (0) gives the field state at leading order in a perturbative expansion, in which it is assumed that the cavity field follows adiabatically the matter-wave dynamics. Furthermore, it is seen that it is a function of the atomic density, which enters both in the numerator and in the denominator of the expression. In this work we will be interested in determining the atomic quantum ground state, which emerges from a coupled dynamics between cavity field and atoms, giving rise to the mechanical potential sustaining such state.
In order to gain further insight into the effect of the cavity potential on the atoms, we consider the semiclassical limit, which can be found from Eq. (11). In this limit, we identify the terms which explicitly depend on the atomic position with mechanical forces. The corresponding potential takes the form The third term on the right-hand-side of Eq. (12) is the a.c.-Stark shift due to the laser field. In the following we assume that the external laser field drives the atoms uniformly and the Rabi frequency Ω does not depend on the position x (i.e. f (x) = 1). The Fourier transform of the potential is hence composed by a term at wavevector 2k whose amplitude is proportional to the number of photons, and which is due to the dispersive coupling between photons and atoms. This term corresponds to a potential with periodicity λ/2, analogous to an optical lattice in free space. Differing from an optical lattice in free space, the number of photons depends on the atomic density via the "bunching parameter". The second component is at wave vector k and thus oscillates with periodicity λ. It originates from coherent Raman scattering of laser photons into the cavity field and its amplitude is hence proportional to the cavity electric field amplitude, but it also depends on the positions of the scatterers through the "spatial order parameter" Θ. The two Fourier components give rise to an effective potential with spatial periodicity λ as displayed in Fig. 1. We remind that potential (12) is found in a semiclassical limit. In this limit, the expression we find agrees with the ones derived in [9,31] within a semiclassical model. The model of a semiclassical potential for the atoms does not take into account dissipative effects, which may be due for instance to cavity decay (while dissipation due to back-action of the resonator over the atoms is here a priori neglected as we assume the adiabatic limit). Indeed, in the dynamics of the atoms we find non-hermitian terms, which are characterized by the position-dependent coefficient scaling with the linewidth κ of the resonator. Correspondingly, input noise gives rise to fluctuations in the height of the potential, which are described by the operator a in (t) in Eq. (10). The effect of cavity quantum noise will be here discarded, assuming that the detuning |∆ c | ≫ κ (more precisely, |∆ c | ≫ κ, |U 0 Y|, which is consistent with the assumption of adiabatic elimination of the cavity degrees of freedom). In this regime, the atoms move in a dispersive potential, which is produced by the dynamical Stark coupling with the cavity mode photons and by the Raman-scattered field of the laser.

C. Tight-binding limit
The motion and steady state of the semiclassical model discussed in the previous section has been studied in several theoretical works. Numerical simulations demonstrated that the system self-organizes, such that the atoms localize at the antinodes of the cavity field, where their coupling is maximum, according to an array with periodicity λ [8]. This behavior appears when the laser intensity exceeds a critical value [8,9], and has been confirmed experimentally [5]. Localization of the atoms may take place around two possible set of minima, one centered at the positions x , while the cavity field is a coherent state with amplitude proportional to the pump [29,30]. In the limit in which the atoms are well localized at the minima one can expand potential (12) till second order in the fluctuations of the particles position. For a particle localized at x (0) j the potential takes the form where the potential due to the other particles enters in a mean-field approach through the mean value of operators Z and Y. Note that when the atoms are self-organized at the set of positions {x , then the sign of Z is positive (respectively, negative).
Consistently with the approximations made so far, the dynamics can be characterized by the dispersive dipolar potential provided that |U 0 Y |, |U 0 Z | ≪ |∆ c |, which allows for the adiabatic elimination of the cavity field from the matter-wave equations. This implies that, for the regime we consider, the sign of this term of the potential is solely determined by the sign of the detuning between cavity and pump. One hence finds that the position x (0) j are minima provided that ∆ c < 0. This behavior can be understood in terms of positive feedback of the system, which is warranted whenever the sign of Z is opposite to the one of the harmonic potential V (2) , and hence when ∆ c < 0. This latter condition leads to the property that the conservative force, due to the potential, is attractive [31]. It is a sufficient condition, provided that the detuning ∆ a between atom and laser is negative, ∆ a < 0 (and consequently U 0 is negative), as it is visible from the numerator of Eq. (14) (note that for the situations here considered, | Z | > Y , see also Ref. [13]). Indeed, in this regime the minima of the potential, formed by light scattering of the atoms, are also minima of the standing-wave potential of the cavity field, so that the attractive forces of both contributions add up to confine the atoms. On the contrary, when ∆ a > 0 the opposite situation is realized: the minima of the potential due to light scattering are now maxima of the cavity standing-wave potential: the term proportional to U 0 in the numerator of Eq. (14) has opposite sign than the detuning ∆ c , and the overall effect is to make the effective potential, resulting from the two contributions, shallower. An extensive discussion on the semiclassical forces as a function of the detunings can be found in Refs. [9,31].
Finally, the amplitude of the semiclassical detuning depends nonlinearly on the atomic density through quantity ∆ c − U 0 Y in the denominator. Bistability effects due to this nonlinearity have been discussed for the situation, in which the cavity is pumped by a laser [17,22,25,26,32,33]. In this article we will focus on the regime where |∆ c − U 0 Y | ≫ κ, when the potential is conservative and dissipative effects can be neglected. This parameter regime is far away from the bistability region. Correspondingly, this term will be treated as a small correction of the semiclassical potential in the framework of perturbation theory.

A. Derivation of the effective Hamiltonian
We now assume that the atoms are tightly confined in one set of minima of potential (12), say, at the position x 2j = jλ. This assumption is clearly based on the semiclassical dynamics, as a potential cannot be simply singled out from Eq. (11). Our aim is to identify the quantum ground state by considering the full quantum dynamics. For this purpose, starting from the assumption of tight confinement, we perform a Wannier decomposition of the atomic field operator, where operator b j annihilates a particle at the site centered in x where N = j b † j b j is the atom number operator, and B = j b † j b j+1 + H.c. is the hopping operator, while the coefficients read and g 1D = g g is the scattering strength in one dimension. In writing Eq. (16) we made the nearest-neighbor approximation, and introduced the chemical potential µ, assuming a grand-canonical ensemble. We remark that the Bose-Hubbard expansion of Hamiltonian (16) relies on localization of the atoms at the minima of the potential inside the cavity. We now use Eq. (15) in Eq. (11), and determine the equations of motion for the operators b l by multiplying with w l (x) and integrating over the position. We finḋ which is written in the regime in which |∆ c | ≫ |U 0 Y|.
which considers only nearest-neighbor coupling and where we used with and ℓ = 0, 1. At first order in the expansion in the small parameter U 0 /|∆ c |, Eq. (19) can be exactly cast in the formḃ with where we consider κ ≪ |∆ c | and we truncated the second term in an expansion at first order in |Z 1 U 0 N/∆ c | and |J 1 U 0 N/∆ c |. Note that in expression (26) the operators are symmetrically ordered, so to obtain Eq. (19). The total dynamics of the system is now rendered by the Hamiltonian where the first term of the right-hand-side, defined in Eq. (16), describes the hopping between sites, due to the quantum fluctuations, and the on-site interaction emerging from s-wave scattering, while the second term contains the contributions due to the mechanical effects of the laser and cavity potential, which determine and sustain the atomic pattern. We note that Hamiltonian (26) is similar to the Bose-Hubbard Hamiltonian, however, its terms are nonlinear in the number and hopping operators. Moreover, it differs also from the usual Bose-Hubbard model due to the appearance of a term proportional to the squared of the hopping operator, B 2 . The nonlinearity in Eq. (26) is due to the coupling with the cavity field, which scales with the number of atoms and which determines the confining potential: the larger is the atomic number the stronger is the coupling. This effect is exquisitely due to cavity quantum electrodynamics, and originates from the constructive interference with which the atoms scatter the laser photons into the cavity mode. We also note that, when considering higher order terms in U 0 /∆ c , an exact effective Hamiltonian -which is a function solely of operators N and B fulfills Eq. (19)cannot be found unless one resorts to a suitable thermodynamic limit [18]. In fact, these terms emerge from the nonlinear coupling between photons and atoms inside the resonator, in which the atoms act as a refractive index. The effect is hence highly nonlocal, and it is significant when the strong coupling regime is warranted. An immediate consequence is that Wannier functions will depend on the particle number. Most remarkable is the fact that the Hamiltonian is quadratic in the hopping operator, even when the nonlinearity deriving from the atomic refractive index, U 0 Y, is negligible. This property is due to the collective scattering of the atoms into the resonator. Nevertheless, it does not affect substantially the property of the ground state at large potential depth (small tunneling rates) as we will show.

B. Mapping to a Bose-Hubbard model
For the purpose of studying the quantum phases of the many-body system, we define a thermodynamic limit. This is here so defined, that the density of the atoms is kept fixed as the particle number N and the cavity-mode volume go to infinity. Denoting by 2K the total number of sites of the lattice, such that in the considered configuration there are K lattice cells, the cavity coupling strength is inversely proportional to the squared root of the cavity mode volume, and thus scales as The other parameters in Eq. (27) scale as U 0 = u 0 /K and S 0 = s 0 / √ K, where u 0 and s 0 are constants, while the number operator is now N = Kn 0 , with n 0 the density operator, giving the number of atoms per site. This scaling is such that, as K → ∞, the parameters in Hamiltonian (27) depend only on the atomic density. We also note that in this thermodynamic limit a Bose-Hubbard form can be also derived when the terms scaling with U 0 are taken into account, see for instance [18] for a discussion. In this thermodynamic limit we rewrite Hamiltonian (27) in the Bose-Hubbard-like form, where and where U , Z j , and J j are functions of n 0 through the Wannier functions. Insight into the dependence of the coefficients on the density can be gained using a (modified) Gaussian ansatz in place of the Wannier functions, as reported in Appendix 1.
Differing from the ordinary Bose-Hubbard model [14], in the case of self-organized atomic patterns the coefficients entering the effective BH Hamiltonian do depend on the atomic density. This property implies that the atomic density is not solely determined by the chemical potential [17,18]. We also observe that, in addition to the nearest-neighbor hopping arising from the kinetic energy, there is a hopping term due to the cavity potential which contains a term proportional to B 2 . This latter contribution describes long-range correlations among the atoms mediated by the scattered photons. Close inspection shows that, in the tight-binding regime, its coefficient is of higher order with respect to the coefficient multiplying B, see App. 1. Within the considered thermodynamic limit, this term gives rise to a small correction to the spectrum of the lowest excitations and corresponding eigenstates of the Mott-insulator states, which must, however, be taken into account when determining the ground state of the system for larger values of the coefficient Z 1 , scaling the tunneling coefficient t 2 and some terms of t 1 .

IV. INCOMPRESSIBLE PHASE OF SELF-ORGANIZED ATOMIC PATTERNS
We now study whether the self-organized atomic pattern exhibits an incompressible ground state. Therefore, we focus onto the regime in which self-organization has set in, namely, when the laser amplitude exceeds the threshold value [9,13] and the laser frequency is smaller than the cavity-mode frequency, ∆ c < 0. The atoms are assumed to be localized at the set of minima {x 2j = jλ}, such that Z 0 > 0.
In order to display the phase diagram for this system, we consider the chemical potential and the amplitude of the pump field s 0 , which indirectly determines the tunneling coefficients. In fact, for s 0 → ∞ the intracavity field potential is deepest and tightly localizes the atoms at their minima, and the tunneling parameters t 1 and t 2 , Eqs. (29) and (30), vanish correspondingly (as one can check by considering the dependence of the coefficients Z j and J j on s 0 , see Appendix 1).
We apply the strong coupling expansion developed in Ref. [19] in order to determine the size of the incompressible states as a function of the pump amplitude and of the chemical potential. For this purpose we start by considering the limit s 0 → ∞, assuming the system to be in a Mott-insulator state with an integer occupation per site n 0 . We calculate the free energy of the Mott-insulator state E M (n 0 ) and of the defect states E ± (n 0 ), which are obtained by adding or removing a particle to the Mottinsulator state. The energy of the defect state is found by means of degenerate perturbation theory, expanding in power of the small parameter t 1 /U up to third order. In the parameter regime we consider, this expansion corresponds also to a first-order expansion in the parameter t 2 . The phase boundaries between the Mott-insulator and the compressible phase correspond to the situation where the energy difference between the defect state and the Mott-insulator state vanishes, namely, when The values of the chemical potentials µ + (n0) and µ − (n0) , at which the Mott-insulator phase with integer occupation n 0 becomes unstable when adding and removing a particle, respectively, fulfill the equations When µ + (n0) > µ − (n0) , then µ + (n0) and µ − (n0) determine the upper and lower boundaries of the parameter region where the system is in the Mott-insulator state. The phase diagrams reported afterwards are computed using the analytical formulae of the strong coupling expansion, Eqs. (33) and (34) in the thermodynamic limit, fixing the values of the parameters u 0 and s 0 . The coefficients are evaluated using a modified Gaussian ansatz in place of the Wannier functions, see Appendix for detail. Figure 2 displays the phase diagram for the considered system, showing three incompressible, Mott-insulator phase regions with n 0 = 1, 2, 3, for ∆ c = −50κ and u 0 = −0.1κ. When plotted as a function of 1/s 0 , the Mott-insulator regions become larger as a function of the laser amplitude as the number of atoms increases. This result can be understood with simple arguments, in fact the trapping potential is constituted of the light coherently scattered by the atoms: As the number of atoms increases, the cavity-field intensity, for a given pump amplitude, is larger, thus the potential becomes deeper and the configuration more stable. Another remarkable property is the appearance of gaps between the Mott-insulator lobes. In the limit of vanishing tunneling coefficients t 1,2 → 0 (corresponding to s 0 → ∞), this behavior implies that the degeneracy among states with different densities, found in the ordinary BH model, is here removed. This behaviour results from the dependence of the single-particle, on-site energy on the average density. In particular, while inside the lobes the density per site n 0 takes integer values, in the gap between the lobes the density changes continuously in the interval r < n 0 < r + 1 (with r = 1, 2, 3, . . . integer number), as shown in Fig. 3. The region outside the Mott-insulator lobes is presumably a peculiar superfluid state, which entails photonic excitations. A study of its properties close to the self-organization threshold has been presented in Refs. [8,13]. The appearance of the gap is understood, when one considers the dependence of the coefficients on the density. In fact, at s 0 → ∞ (corresponding to putting the tunneling elements to zero), the Mott-insulator region with n 0 = 1 is found for values of µ such that 0 < µ 1 < U 1 , where µ 1 = µ[n 0 = 1] and U 1 = U [n 0 = 1] (see Eq. (18)). Similarly, the region at n 0 = 2 is found for U 2 < µ 2 < 2U 2 (with U 2 = U [n 0 = 2]). The relation U 1 < U 2 (U j < U j+1 ) leads to the appearance of parameter regions where the system is compressible even at zero tunneling. This relation is a consequence of the fact that, for a fixed amplitude of the laser field, the intracavity field becomes larger as the number of atoms (scatterers) is increased. Figure 4 displays different phase diagrams, obtained for various values and signs of the detuning ∆ a , keeping ∆ c fixed. We first focus on the case in which ∆ a < 0, when the potential due to the dynamical a.c.-Stark shift, induced by the coupling with the cavity mode, adds up to the potential due to light scattering, giving rise to a stronger attractive dipole force. In this case, we observe that for larger values of u 0 the gap between the chemical potentials of Mott-insulator states at different densities increases. Moreover, the larger is u 0 , the larger is the area covered by Mott-insulator states at larger densities: incompressible states are found at smaller values of the laser amplitude. The opposite tendency is found when ∆ a > 0, and hence when the potential due to the a.c.-Stark shift competes with the scattering potential. In this case, as |u 0 | is increased, the gap between the Mottinsulator lobes becomes smaller, while the area covered by Mott-insulator states with larger densities decreases. In particular, incompressible states are observed at larger values of the laser pump amplitude. This behavior can be understood from the dependence of the tunneling coefficient, Eq. (29), on the various terms: The term proportional to u 0 scales with the onsite density, and it hence becomes more important as n 0 is increased. For values of the detuning, such that the cavity potential adds up to the scattering potential (∆ a < 0), at a fixed laser intensity the confinement is larger for a larger density of atoms. Correspondingly, the parameter region where the system is compressible becomes larger. On the contrary, for ∆ a > 0 the cavity potential tends to make the scattering potential shallower, the confinement tends to depend less strongly on the density and the gap between the Mott-insulator lobes is smaller. 1κ. The arrow indicates the threshold value of the laser amplitude, for which the system self-organizes [13]. The chemical potential is here reported in units of the on-site energy U1, corresponding to the value of Eq. (18) for the density of one atom per site, with g1D/(ERλ) = 3.74 · 10 −4 and ER the recoil energy, considering a gas of 87 Rb atoms whose dipole-transition, at wavelength λ = 830 nm, is coupled to the cavity mode.
We remark that, while in this model atom-atom interaction is essentially s-wave scattering, it should be considered that, when the atomic occupation of each site exceeds unity, then dipole-dipole interaction, including superfluorescence, will be relevant. In our model we neglect collective effects in the spontaneous decay outside of the cavity. In particular, for the parameter regime we consider, incoherent processes are suppressed for the large atomic detunings we choose. On the other hand, one can identify parameter regimes where coherent coupling between the atoms at a site may be relevant, and which should be included in an effective onsite interaction term.
We finally estimate the parameter regime required for observing our predictions, considering the setup reported in Ref. [26]. Here, ultracold 87 Rb atoms are loaded inside a resonator, where a mode couples quasi-resonantly with the transition at wave length λ = 780 nm. The parameters are g 0 /π = 14.1 MHz, κ/2π = 1.3 MHz and γ/2π = 3 MHz. Taking |∆ a |/2π = 60 GHz, Ω/2π = 60 MHz, we find U 0 /2π ∼ 0.03 MHz and S 0 /2π = 0.1 MHz, while the effective rate of spontaneous emission is of the order of few KHz. For a number of atoms N ∼ 10 4 , the relation S 0 √ N ≫ κ is fulfilled, warranting the existence of a time-scale in which the atoms may experience a conservative potential formed by their scattered photons. The relation |∆ c | ≫ |U 0 |N, κ is satisfied taking |∆ c |/2π = 100 MHz. For these parameters, the value s 0 /κ = 10 at which incompressible phases are found in Figs. 2 and 4, corresponds to S 0 √ N ∼ 10κ. This latter value corresponds to the one found for the experimental parameters in [26], showing that the regime, where quantum effects in self-organized atomic patterns are visible, can be accessed with existing experimental setups.

V. CONCLUSIONS
The quantum ground state of a self-organized atomic pattern, confined by the potential of the field scattered into the cavity mode, can be incompressible provided that the laser intensity exceeds a certain critical value, which is found above the threshold value at which the system self-organizes. In addition, intervals of values of the chemical potential are identified, in which the state is compressible even when the tunneling vanishes. These appears as gaps between the Mott-insulator lobes in a phase diagram we derive, and their origin can be understood from the competition between confinement, which is steeper at large atomic density for a given laser intensity, and interparticle collisions, whose effective strength also depends on the density via the localization of the atomic wave function in the intra-cavity field potential. Our analysis is based on an effective Bose-Hubbard model, according to the assumption that the atoms are tightly bound at the minima of the intra-cavity field potential. Since the height of such potential depends on the atomic density, the coefficients of the model depend on the atomic density giving rise to the peculiarities encountered in the phase diagram. Comparison with typical parameters of experimental setups, dealing with ultracold atoms inside of optical cavities, shows that the prediction here made could be experimentally tested in existing setups.
Detection of the quantum properties of the system may be performed by measuring the light using a probe field [6,34,35], in a direction of emission corresponding to a Bragg scattering angle. A spectral analysis of the probe intensity will provide information on the state of the system, which can be studied as a function of the laser pump intensity, and hence of the tunneling rate [36,37]. Sweeping the phase diagram in the directionμ may be performed in various ways. One possibility is to implement a shift of the atomic ground state by means of a magnetic field or by off-resonant coupling with a third electronic state.
In this work we focused on the quantum ground state of the self-organized structure, arising by driving the atoms with a strong, transversal laser field. The mechanical potential results from the scattering potential, with periodicity equal to the wavelength λ, and from the potential due to the cavity field mode, with periodicity equal to the wavelength λ/2. The regime we considered is the one in which the scattering potential is the strongest. In this regime, it is expected that the light at the cavity output is coherent, while the intensity is independent on the number of atoms when the strong-coupling regime is warranted [29,30]. An interesting question is what are the properties of the emitted light when the cavity-field mode potential dominates. In this regime, in a semiclassical model it has been found that the light at the cavity output can be squeezed, and in general anti-bunched [38,39]. An intriguing question is whether and how these properties are modified, when the atomic quantum statistics is relevant to the dynamics of atom-photon interactions.
The solution of the Bose-Hubbard model requires the evaluation of the coefficients, which are integrals containing Wannier functions. The Wannier functions are solutions of the Schrödinger equation for a single particle in presence of the potential in Eq. (12), which itself depends on the spatial distribution of the atoms inside the cavity. The problem is hence nonlinear. We solve it by combining an analytical and a numerical approach. In first instance, we replace the Wannier functions by modified Gaussian functions, w i (x) →w i (x), with w i (x) = 1 + 3δ 2 4 g i (x) − δ 2 (1 + 3 2 δ 2 ) (g i+1 (x) + g i−i (x)) , and is a normalized Gaussian function of width σ. The form of the modified Gaussian function in Eq. (35) warrants orthonormality up to second order in δ, where δ = dxg i (x)g i+1 (x) = e −π 2 /(k 2 σ 2 ) and where the overlap between second neighbors is of the order O(δ 4 ). We remark that the coefficient in (35) at second order in δ warrants the correct normalization. The width σ of the Gaussian function g i (x) is found by minimizing the single-particle energy E, for the Schrödinger equation in presence of potential (12). For later convenience we introduce the dimensionless parameter which is y ≪ 1, consistently with our assumptions. In terms of the parameter y this consists in finding the min-imum of the equation where E R =h 2 k 2 /2m is the recoil energy and we introduced the functions whereṼ 1 andṼ 2 arẽ Note that the cos-functions in Eq. (12) have been expanded up to second order in the displacement from the minima. Minimization with respect to y leads to the nonlinear equation with F ′ (y) = dF (y)/dy and G ′ (y) = dG(y)/dy, and which is solved numerically by recursion.
Using the value which minimizes the single-particle energy, y := y min , we evaluate the coefficients we need in order to study the problem. They are reported below as a function of y: where E R =h 2 k 2 /2m is the recoil energy and we discarded terms of higher order than δ 2 . Note that the coefficients Z j are here reported for the pattern localized at the positions x