{'fraction_non_alphanumeric': 0.04533281394556167, 'fraction_numerical': 0.016263329600233724, 'mean_word_length': 4.40189374013677, 'pattern_counts': {'":': 0, '<': 10, '<?xml version=': 0, '>': 8, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 8, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Limit Cycles in Replicator-Mutator Dynamics with Game-Environment Feedback

This paper considers the coevolutionary game and environment dynamics under mutations of strategies. Individuals’ game play affects the dynamics of changing environments while the environment in turn affects the decision-making dynamics of individuals through modulating game payoffs. For some such closed-loop systems, we prove that limit cycles will never appear; however, in sharp contrast, after allowing mutations of strategies in these systems, the resulting replicator-mutator dynamics under environmental feedback may well exhibit Hopf bifurcation and limit cycles. We prove conditions for the Hopf bifurcation and thus the existence of stable limit cycles, and also illustrate these results using simulations. For the coevolutionary game and environment system, these stable limit cycles correspond to sustained oscillations of population’s decisions and richness of the environmental resource.


INTRODUCTION
Recently in the theoretical study of evolutionary games, the integrated model of games and their environment feedback has attracted researchers' attention. In the classic game setting, the payoffs in each pairwise-interaction game are usually predetermined and given in a form of constant payoff matrices. However in many applications, especially in the context of shared resources, it is recognized that the payoffs for individuals can change over time or be affected directly by the external environment. Thus gameplaying individuals' decisions can influence the surrounding environment, and the environment also acts back on the payoff distributions. This mechanism, which is termed game-environment feedback, has been studied in biological and sociological models [Weitz et al. (2016), Lee et al. (2019)].
The well-known replicator dynamics model [Sandholm (2010)] has been used to study the coupled game and environment dynamics. Interesting system behaviors, such as periodic orbits and heteroclinic cycles, have been revealed by incorporating the dynamic payoffs into the two layers of the replication system. Although the replicator dynamics have been proved to be a powerful model in analyzing a variety of classical games from an evolutionary dynamics perspective, they do not take the mutation into account, The work of L. Gong, W. Yao, and J. Gao  which is a key component of natural selection theory [Page and Nowak (2002)]. Generally mutations can be captured by allowing individuals to spontaneously change from one strategy to another in small probability. This yields the so-called replicator-mutator dynamics [Komarova (2004)], which also have played a prominent role in evolutionary game theory and appeared in a variety of contexts in biology and sociology. Their applications include, but are not limited to, populations genetics [Hadeler (1981)], language evolution [Nowak et al. (2001)], social decision evolution [Lee et al. (2019)], and multi-agent network interactions [Pais et al. (2013)].
After the game-environment feedback was first introduced to study the coevolution of a single population and its environment [Weitz et al. (2016)], different extensions and variations have appeared. In [Hauert et al. (2019)], researchers have considered the case with more general asymmetric payoff matrices and shown rich dynamics from an eco-evolutionary perspective. It has been considered in [Tilman et al. (2020)] that different environmental resource model can be exploited in addition to the original logistic model, such as renewable resource and decaying resource, and it was found that limit cycles instead of neutral periodic orbits can appear under specific conditions. Other works [Gong et al. (2018), Kawano et al. (2019), Muratore and Weitz (2019)] have extended the framework to the two-population case where two different populations interact and coevolve under different feedbacks from the environment. It was shown that neutral periodic oribits exist and convergence to the boundary happens in higher dimensional systems. In most of the existing studies within the game-environment framework, little attention has been paid to the effect of mutations in strategies. We also note that although rich system behaviors, such as convergence to the equilbiria or heteroclinic cycles on the boundary and existence of neutral periodic orbits, have been identified, the results on the limit cycle dynamics have been rarely reported.
In this paper we develop a 2-dimensional replicatormutator model with game-environment feedback and prove that limit cycles cannot appear in the low dimensional coevolutionary dynamics without mutations; the heteroclinic cycles or neutral periodic orbits are the most marginal behaviors that the system can exhibit. Then, we allow mutations therein and prove the conditions of Hopf bifurcation and thus the existence of the limit cycle. The type of the bifurcation is supercritical such that the generated limit cycle is stable. Mathematically speaking stable limit cycles imply sustained oscillations: any small perturbation from this closed trajectory can only lead the system to return to it, and so the system sticks to the limit cycle. For the studied coevolutionary dynamics here, the stable limit cycles may have the new interpretation of persistent oscillating strategy switching, which helps one gain new insight into the classic dilemma of the tragedy of the commons.
The rest of the paper is structured as follows. Section 2 introduces the environment-dependent payoffs and replicator-mutator equations, and formulates the mathematical model with game-environment feedback. Section 3 establishes the non-existence of limit cycles, and Section 4 analyzes the Hopf bifurcation process and stability of the limit cycles and also illustrates the results using simulations. Conclusions are drawn in Section 5.

Environment-dependent payoffs
When games are played in a changing environment, people usually choose to characterize the environment by the richness r ∈ R of a resource of interest; such a resource has important influence on the payoffs that players receive in each pairwise-interaction game. Correspondingly, the payoff matrix becomes dynamic and dependent on r. The individuals tend to cooperate in a situation with abundant environmental resource, and become more selfish and choose to defect while the resource becomes depleted. Following Weitz et al. (2016), we assume that the environment-dependent payoff matrix is in the form of linear combination of the payoff matrices from the classic Prisoner's Dilemma, i.e., where to realize mutual cooperation and mutual defection as the Nash equilbiria respectively associated with the two payoff matrices, the constant elements in the matrices satisfy R 1 > T 1 , S 1 > P 1 , R 2 < T 2 and S 2 < P 2 .

Replicator-Mutator equations
In a well-mixed and infinite population, the individuals play the Cooperation-Defection (C-D) game with the pay-off matrix A(r). We denote the proportion of individuals choosing C by a variable x. Since there are only two strategies and x ∈ [0, 1], the population state can be represented by the vector x = [x 1 − x] T . Assume the strategies can mutate into each other with the same probability µ ∈ [0, 1]. Thus, the dynamics of x are governed by the following replicator-mutator equatioṅ where (A(r)x) 1 is the first entry of A(r)x and represents the fitness of choosing strategy C, and x T A(r)x is the average fitness at the population state x.

Mathematical model with game-environment feedback
To model the evolution of the environmental change, we use the standard logistic model which is given bẏ where θ > 0 represents the ratio between the enhancement effect due to cooperation and degradation effect due to defection. Combining (2) and (3), we obtain a closed-loop planar system describing the population dynamics under the game-enviroment feedback . (4) The state space of the system, which is a unit square I = [0, 1] 2 with the boundary ∂I (four sides), is invariant under the dynamics (4). The interior of this square is denoted by int I = (0, 1) 2 .
Before going into the main results, we first list two theorems that will be referred to later. Theorem 1. (Poincaré-Bendixson theorem). [Hofbauer and Sigmund (1998)] Letẏ = f (y) be a planar system of differential equations defined on an open set U ⊆ R 2 . Let ω(y) be a nonempty compact ω-limit set. Then, if ω(y) contains no rest point, it must be a periodic orbit.
The Poincaré-Bendixson theorem delineates all possible limiting behaviors of a planar system. As a consequence of this theorem, one immediately has the following corollary. Corollary 2. If Γ is a periodic orbit that forms the boundary of an open set Ω, then Ω contains an equilibrium point. Theorem 3. (Bendixson-Dulac criterion). [Wiggins (2000)] For the planar systemẏ = f (y) defined on a simply connected region U ⊆ R 2 , if there exists a C 1 function ϕ(y) such that the divergence of ϕf , div(ϕf ) = ∂(ϕf1) ∂y1 + ∂(ϕf2) ∂y2 , is not identically zero and does not change sign in U, then the system admits no periodic orbits.
The Bendixson-Dulac criterion is an important technique for proving that periodic orbits do not exist. The C 1 function ϕ(·) is called the Dulac function. In the following two sections, we present our main results.

NON-EXISTENCE OF LIMIT CYCLES WITHOUT MUTATIONS
In this section, we consider the case when there are no mutations in the strategies which reduces to the setting in Weitz et al. (2016), where it has been shown that limit cycles do not occur without a formal proof. We will use the Bendixson-Dulac criterion to prove this result formally.
In the absence of mutations, namely when µ = 0, after substituting (1) into system (4), we obtain where for brevity, we denote a = R 1 − T 1 , b = S 1 − P 1 , c = T 2 − R 2 , and d = P 2 − S 2 . All of these parameters are positive under the setting of (1).
This system has four equilibria on the corners of the square I and an additional equilibrium determined by equations Solving these two equations, one obtains the equilibrium (x * , r * ) One can check that this is the interior equilibrium since (x * , r * ) ∈ int I. Now we present our first result. Theorem 4. The system (5) admits no isolated periodic orbits in all its parameter space.
Proof. First, note that the boundary ∂I itself is invariant under (5) and the four corners are equilibria, then the boundary of I does not contain any periodic orbits. So it suffices for one to focus on the interior int I. If the system does not have any periodic solutions in int I, the theorem holds trivially.
Now consider the case when there is a periodic solution for (5) in int I denoted by Γ. Since there is only one equilibrium in int I, according to Corollary 2, this equilibrium must be in the interior region of Γ. We define the Dulac function ϕ(x, r) = x α−1 (1 − x) β−1 r γ−1 (1 − r) δ−1 (8) with coefficients α, β, γ, and δ which will be specified later. Note that this function is strictly positive in int I. Denote the right hand side of the first equation in (5) by f 1 (x, r), and f 2 (x, r) for the second equation. Then we compute the divergence of the vector field (ϕf 1 (x, r), ϕf 2 (x, r)) One can choose α, β, γ, and δ such that the following equations are satisfied To show the existence of such α, β, γ, and δ , is equivalent to showing that equations (10) have solutions taking α, β, γ, and δ as unknown variables. Equations (10) are linear and non-homogeneous, and the coefficient matrix and augmented matrix are C and [C|D] with One can easily check that rank C ≡ rank [C|D] ≤ 4, which ensures that (10) have at least one set of solutions.
Then (10) yields Should the periodic orbit Γ exists, the Bendixson-Dulac criterion would imply that either ϕ(x, r)(αb − γ) is identically zero or it changes sign in int I. Because ϕ(x, r) is strictly positive, we have αb − γ = 0. Then (11) turns out to be According to the theorem about First Integral in Appendix A, the specific function ϕ(x, r) now serves as an integrating factor of (5), such that system (5) has a first integral in int I given by The derivative of V (x, r) over time t satisfies = ϕf 1 f 2 − ϕf 1 f 2 ≡ 0, and hence V (x, r) remains constant along the solutions of (5). Now we are ready to prove the result by contradiction. Suppose the periodic orbit Γ is isolated in int I, i.e., there are no other periodic orbits within some neighborhood of Γ. Then every trajectory beginning sufficiently close to Γ spirals toward it either as t → +∞ or as t → −∞ (Lebovitz, 1999, Theorem 7.4.3). Consider an arbitrary trajectory spiraling toward Γ as t → +∞ (by reversing time the other case can be handled). Assume it starts from the initial point (x 0 , r 0 ) and denote the trajectory by φ(x(t), r(t)). Then from (Hirsch et al., 2004, Corollary 10.1), one knows that (x 0 , r 0 ) must have a neighborhood Ω such that Γ is the ω-limit set for all points in it. We have shown that system (5) admits a first integral when there is a periodic orbit in int I. Let κ ∈ R be the constant value of V on Γ, then one has V (φ(x(t), r(t))) ≡ κ in view of continuity of V . Since all solutions in Ω spiral toward Γ, thus V is constant on Ω, which means its partial derivatives are zero in Ω. One can calculate the partial derivatives of V (x, r): Since ϕ(x, r) is positive in int I, the derivatives only equal zero at the interior equilibrium (x * , r * ). This leads to a contradiction. Therefore, when there is a periodic orbit in I under system (5), it cannot be isolated. 2 The result of Theorem 4 can be further specified with regard to limit cycles. For planar systems of the forṁ y = f (y), a limit cycle is an isolated periodic orbit Γ that is the α-limit set or ω-limit set for some y / ∈ Γ. From Theorem 4, system (5) does not allow the possibility of isolated periodic orbits, thus one can immediately claim the non-existence of limit cycles in system (5) by the definition. Corollary 5. The dynamics under (5) cannot exhibit limit cycle behaviors.
Note that this statement only concerns limit cycles. Other limiting behaviors, such as neutral stable periodic orbits or heteroclinic cycles on the boundary, may exist in some scenarios [Weitz et al. (2016)].

Hopf Bifurcation and Limit cycles
In the previous section, we have proved that there are no limit cycles for system (4) when there are no mutations. In this section, we consider the effect of mutations and study further if limit cycles can appear in this case, and if yes, what their stability is.
We allow mutations to happen with a probability µ ∈ (0, 1]. Notice that in the expression ofṙ, the coefficient of x, (θ + 1), is always larger than 1, and it only affects the equilibrium's position. For simplicity of computation, we fix θ = 1, and arrive at the following system . (14) Note that the corners of I are not equilibria anymore, and thus we have to calculate the possible equilibria again for this new system. The equilibria are obtained by setting the right hand sides to be 0, i.e., Solving these two equations, one can obtain the possible equilibria. There are some equilibria which always stay on the boundary for all possible parameters values. We do not go into the details of them now since limit cycles could never occur on the boundary. Now, let us focus on the interior equilibria.
By calculation, we obtain that the system (14) has a unique interior equilibrium q * = ( 1 2 , a+b a+b+c+d ). Now we evaluate the Jacobian of the vector field f (x, r) at the interior equilibrium q * , which is where with µ 0 = ad−bc 4(a+b+c+d) . Notice that J 12 < 0 and J 21 > 0, and the Jacobian matrix's trace and determinant are tr(J(q * )) = J 11 , det(J(q * )) = −J 12 J 21 > 0.
And the eigenvalues can be calculated We take µ as the bifurcation parameter to study the effect of different mutation rates on the system dynamics. Lemma 6. When µ 0 ≤ 0 (resp. µ 0 > 1), the equilibrium q * of the dynamics (14) is locally asymptotically stable (resp. unstable) for µ ∈ (0, 1], and the Hopf bifurcation can only occur when 0 < µ 0 ≤ 1.
Proof. By examining the Jacobian matrix, one can analyze the stability of the interior equilibrium easily. When µ 0 ≤ 0 (resp. µ 0 > 1), the equilibrium q * of the dynamics (14) is locally asymptotically stable (resp. unstable) for µ ∈ (0, 1]. When 0 < µ 0 ≤ 1, J 11 may become zero as µ changes. If µ = µ 0 , the eigenvalues are a purely imaginary conjugate pair, which is the necessary condition of Hopf bifurcation. 2 Next we will investigate if Hopf bifurcation indeed occurs. If yes, one also needs to identify the bifurcation point and the bifurcation's type. Theorem 7. If the parameters satisfy then the interior equilibrium q * of system (14) with bifurcation parameter µ undergoes a supercritical Hopf bifurcation at µ = µ 0 , which leads to a stable limit cycle for µ < µ 0 in the vicinity of µ 0 .
Proof. The main idea of the proof is to check the conditions of the Hopf birfurcation [Guckenheimer and Holmes (2000)] for the two-dimensional system (14). From Lemma 6, we require 0 < µ 0 ≤ 1, which implies 0 < ad − bc ≤ 4(a + b + c + d).
One can check that the eigenvalues satisfy the following conditions: The first and second conditions together imply that the specified eigenvalues λ(µ 0 ) are purely imaginary. And the third condition asserts that the complex eigenvalues cross the imaginary axis at a nonzero speed. Then, according to the Hopf bifurcation theorem in Guckenheimer and Holmes (2000), one can conclude that the dynamics (14) undergo a Hopf bifurcation at µ = µ 0 and there will exists a limit cycle encircling the interior equilibrium for µ < µ 0 in the vicinity of µ 0 .
To determine the stability of the limit cycle arising from the Hopf bifurcation, we need to calculate the first Lyapunov coefficient 1 (µ) at q * when µ = µ 0 . Following the typical computation procedure in Appendix B, we obtain which is negative since (ad−bc) > 0. Then the equilibrium q * is asymptotically stable for µ ≥ µ 0 (weakly stable at µ = µ 0 ) and unstable for µ < µ 0 . Thus, the limit cycle is stable. Therefore, the system (14) undergoes a supercritical Hopf bifurcation [Guckenheimer and Holmes (2000)]. 2
One can verify that the condition (19) is satisfied and thus the Hopf bifurcation point of the interior equilibrium is µ 0 = 0.1543. Figure 1 shows the phase portraits of the dynamics (14) with the above payoff matrices. In Figure 1. (a), a limit cycle exists around the unstable interior equilibrium and it is almost globally stable for the whole domain except for the upper and lower sides and the interior equilibrium. In Figure 1. (b), the limit cycle disappears and the interior equilibrium becomes asymptotically stable.  (14) with payoff matrices as shown. The red stars identify the unique interior equilibrium q * = (0.5, 0.6809). The blue and green curves are two different trajectories with certain initial conditions. (a). Existence of stable limit cycle when µ = 0.08. (b). Disappearance of limit cycle when when µ = 0.4.  (14) with given payoff matrix. The x-axis is the mutation rate µ, the blue and red curves are stable and unstable equilibria respectively, and the magenta curves show the shape of stable limit cycles.
left and bottom right represent the equilibria on the boundary, and they are always unstable. The curve in the middle which is red and then blue represents the interior equilibrium, and it switches its stability at the bifurcation point µ = 0.1543. As a counter example, Figure 2. (b) shows a plot when condition (19) is not satisfied such that Hopf bifurcation does not occur.

CONCLUSIONS
We have investigated the coevolutionary dynamics of the game and environment when strategies' mutations are taken into account. We used the replicator-mutator model and considered a uniform mutation rate for both strategies. By using Poincaré-Bendixson theorem, Bendixson-Dulac criterion, and Bifurcation theory, we proved the non-existence of limit cycles if there are no mutations; we showed that the system at the interior equilibrium undergoes a Hopf bifurcation which generates a stable limit cycle when mutations are allowed. Our findings suggest the crucial impact of mutations, and the result on such robust oscillations may provide valuable insight and solidly theoretical support for relevant studies in biology and sociology. For future work, it seems to be an interesting direction to study how to properly control the system to behave as desired in [Riehl et al. (2018)].