{'fraction_non_alphanumeric': 0.0337764724068635, 'fraction_numerical': 0.008849899520791467, 'mean_word_length': 4.913391224862888, 'pattern_counts': {'":': 0, '<': 2, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Drift reversal in asymmetric coevolutionary conflicts: Influence of microscopic processes and population size

The coevolutionary dynamics in finite populations currently is investigated in a wide range of disciplines, as chemical catalysis, biological evolution, social and economic systems. The dynamics of those systems can be formulated within the unifying framework of evolutionary game theory. However it is not a priori clear which mathematical description is appropriate when populations are not infinitely large. Whereas the replicator equation approach describes the infinite population size limit by deterministic differential equations, in finite populations the dynamics is inherently stochastic which can lead to new effects. Recently, an explicit mean-field description in the form of a Fokker-Planck equation was derived for frequency-dependent selection in finite populations based on microscopic processes. In asymmetric conflicts between two populations with a cyclic dominance, a finite-size dependent drift reversal was demonstrated, depending on the underlying microscopic process of the evolutionary update. Cyclic dynamics appears widely in biological coevolution, be it within a homogeneous population, or be it between disjunct populations as female and male. Here explicit analytic address is given and the average drift is calculated for the frequency-dependent Moran process and for different pairwise comparison processes. It is explicitely shown that the drift reversal cannot occur if the process relies on payoff differences between pairs of individuals. Further, also a linear comparison with the average payoff does not lead to a drift towards the internal fixed point. Hence the nonlinear comparison function of the frequency-dependent Moran process, together with its usage of nonlocal information via the average payoff, is the essential part of the mechanism.


Introduction
Biology offers a rich laboratory of various types of oscillatory, chaotic and stochastic dynamics. Recently cyclic evolutionary dynamics has been observed in E.coli in vitro [1] and in vivo [2], and attracted interest as a possible mechanism to stabilize biodiversity. This contributes to a long-standing debate how the emergence of new mutants is maintained in biological evolution: Cyclic dynamics has been one of the first proposals for such mechanisms [3]. Cyclic coevolution is not only observed in asexual reproduction. A prominent observation of cyclic domination is observed in side-blotched lizards [4,5]. Three territorial mating behaviour strategies of the male lizards occur, and coincide genetically with orange, blue and yellow blotches. While the cyclic dynamics of rock-paper-scissors type can be demonstrated without taking males and females into account explicitely, hereby an improved quantitative understanding is possible [6].
Social and economic systems are a likewise interesting class of systems in which cyclic dynamics is observed. Social individua deciding in economic situations [7,8,9,10,11,12,13] can fall into oscillatory cycles, e.g. when loners, not participating in the game, are added as a third strategy to a Prisoner's Dilemma [14,15]. Evolutionary game theory is a unifying approach for such systems [16,17,18].
In this paper, a paradigmatic cyclic game is analyzed, which is of likewise importance, for biological mating behaviour, as well as in human social decision dynamics: Dawkins' "Battle of the Sexes" (BOTS), a 2×2 bimatrix game, is the simplest possible type of a cyclic game played by individuals between two homogeneous, mixed populations [19]. In the mating behaviour of females and males, a cyclical dominance of 'slow' and 'fast' strategies can lead to a "Battle of the Sexes", or oscillations, in the infinite population replicator dynamics [19,20,21]. The inherent stochasticity in a finite population [22,23,24] refines this picture, depending on population size and underlying process [25,26,27,28,29,30].
The aim of the paper is to investigate in detail the drift reversal firstly reported in [25], and to corrobate the simulations with an analytic result, allowing for a refined insight into the drift reversal. The main results are (i) that for a generic class of pairwise comparison processes a drift reversal does not occur, and (ii) that for the Moran process, the combination of both a nonlinear reproductive fitness and the comparison with a global function (average fitness) are necessary for a drift reversal.
The paper is organized as follows. In Section 1, the BOTS payoff matrices are introduced and the infinite population description of evolutionary game theory by the replicator equation is recalled. In Section 2, evolutionary birth-death processes are defined in a unifying framework for comparison. In Section 3 the influence of the stochasticity on the time evolution of the population densities is motivated by computer simulations of the process.
In the main Section 4, the average drift (more formally introduced in Sect. 4.1) of the BOTS dynamics is calculated explicitely in finite populations, and the population size corrections are obtained to first and second order analytically for four microscopic interaction processes, for neutral evolution as well as for two linear processes, the Local Update and the linearized Moran process. The paper concludes by discussing and summarizing the results.

Battle of the Sexes: Replicator dynamics
This is the simplest cyclic game between two populations. Following Dawkins [19], the elementary payoffs read for agents interacting with a population of (x, y, 1−x, 1−y) = (i/N, j/N, (N − i)/N, (N − j)/N ) agents in the strategies (π | A , πÃ, π | B , πB). As the opponent's payoff matrix is not the simple transpose of the proponent's payoff matrix, such games are called bimatrix games or asymmetric conflicts. While in the replicator equations [20] picture the population cannot go extinct due do lack of discreteness, for the stochastic description in this paper the population size will be fixed to N female and N male individuals.
For the relative frequencies, or abundance densities, evolutionary game theory, with the implicit assumption of an infinite population, considers the replicator equatioṅ For the standard parameter choice (equivalent to "Matching Pennies") of the BOTS the replicator equation has a constant of motion The replicator equations exhibits neutrally stable oscillations around the x = y = 1/2 fixed point; the adjusted replicator equation (see [20]) has an attractive stable fixed point. In so-called asymmetric conflicts (bimatrix games) where members of the two (sub)populations can receive different payoffs, both populations may gain different average payoffs, and the denominators in the adjusted replicator equations (being the proper N → ∞ limit of the Moran process, see [27]) are different, as π | = − π~ . Thus the denominators cannot be absorbed into a dynamical rescale of time (velocity transformation) and both types of replicator equations not necessarily exhibit the same stability properties. How is this behaviour changed, and how far is it preserved in finite populations? This is the central question addressed in this paper.

Evolutionary processes
To study dynamics in finite populations, it is advised to go down to the microscopic interactions, and to derive macroscopic equations of motion herefrom, eventually utilizing a finite-size expansion to derive fluctuation corrections to the deterministic limit. As the microscopic dynamics may depend on the system at hand, the respective biological or behavioral setup may require different interaction and competition processes. These, however, can be cast into a unifying framework. Following previous investigations [7,25], we consider two classes of birth-death processes: the frequency-dependent Moran process [22,23,24] describing competition with the whole population, and local twoparticle interaction processes, with linear [25] or Fermitype [15,31,32] dependence of the reproductive fitness as a function of the payoff difference between two competing agents. For all processes, the payoffs for an individual read

Moran process
The Moran process [22], a birth-death process, thus preserving N , is a standard model of mathematical genetics describing random inheritance in overlapping generations. In its original formulation, fitnesses of the genetic types were independent of abundance densities in the population, i.e., coevolution was not taken into account. In the frequency-dependent Moran process [23,24], each individual competes with the whole population, or an representative fraction of it, and reproduces proportional to this (cumulative) payoff normalized by the average payoff in the population. With the payoffs averaged over the respective population, for given (i, j) the transition probabilities of the possible four hopping events are given by For better comparison with the processes below, an additional factor 1/2 was introduced. For this commonly used version of the frequency-dependent Moran process one has to ensure that negative payoffs do not lead to negative transition probabilities. This inconsistency is avoided by delimiting w such that the denominator remains positive.

Local processes
In many cases a competition of a single individual with the whole population may be unrealistic. Therefore, it is advised to consider a local competition among individuals. One process of this type is the Local update process [25], where one individual b is selected randomly for reproduction, compares with another randomly chosen individual a, and changes strategy with probability 1 2 (1+w(π a −π b )). The frequency-dependent Moran process (MO), and the other processes below (LU = local update [25], FP = Fermi process [32], and a linearized Moran process (LM) considered below) can more conveniently be written by means of a reproductive function so that the hopping probabilities are for the pairwise comparison processes (LU,FP), and for the other processes (MO,LM) where an individual's payoff is compared to the payoff averaged over the whole population. For all processes, Φ considers a two-particle (birthdeath) process where an individual with fitness π a compares (Eqs. 11-12) with a representative sample of the population (i.e. the average fitness π ), or (Eqs. [13][14] with another individual π b . The frequency-dependent Moran process [22,23], the Local Update [25], and the Fermi process [32] are microscopic evolutionary processes discussed recently. The Linearized Moran (LM) process arises as approximation of the Moran (MO) process in the weak selection limit w → 0.
Note, that elsewhere for the Local update w is replaced by w/∆π max , and for the Moran process, w may not exceed 1 − π min . In the above notation, in the limit w → 0 the Fermi process approaches the Local update process.

Stochastic motion around the fixed point
Having defined different possible microscopic update processes, one desires to gain an intuitive understanding of the resulting stochastic motion, in comparison of small (here, N = 50) and large (here, N = 10000) populations. Figure 1 compares the time evolution of the evolutionary trajectory in phase space (x, y) and Figure 2 in the time evolution of H. For a large population, the qualitative dynamics of the adjusted replicator equation is recovered, i.e., convergence to the internal fixed point and thereby decrease of H which serves as Lyapunov function in the deterministic limit. The motion of the small population is comparatively more stochastic, and after few generations, fixation to the border is reached. Intuitively, the "contracting force" of the stable fixed point is to weak to ensure the observation of a metastable state. Figure 3 shows a longer time interval and four different population sizes.
This paper addresses not only the influence of the population size, but also the influence of the microscopic update process. Figure 4 demonstrates that on longer time scales (here, the time scale is shown on a logarithmic scale to show the convergence behaviour). While the Local Up- date process performs a specific type of random walk in H (in the sense that the internal fixed point is of neutral stability), the Moran process first confines the motion in the vicinity of the attracting internal point, but due to the discreteness of the process, fixates to the border in the limit t → ∞. The initial conditions are the same as in the previous figures, starting from a state different from the fixed point, to show that the Moran process indeed leads to some type of convergence to the metastable state.

Deterministic and stochastic equations of motion at the level of the diffusion approximation
In many cases population sizes are such small that deviations from a deterministic population density cannot be neglected -especially because a species or genetic trait can go extinct, or likewise, a new mutant can either extinct or fixate (i.e., become the common trait in the whole population). If populations are not too small, e.g. N = 10 2 , one however can think of a perturbation expansion in orders of 1/N to derive stochastic corrections to the deterministic equations of motion. A classic approach in mathematical genetics in this direction is the so-called diffusion approximation [33,34], which allows to derive fixation probabilities and densities from a Kolmogorov forward, or Fokker-Planck, equation. For a fixed population size and one trait (higherdimensional cases are prohibitive), the equivalence to a one-dimensional Markov chain allows for a closed exact expression. A similar framework applies for deriving the equations of motion in zero and first order in 1/N , and neglecting higher orders; i.e., we operate at the level of the diffusion approximation. Such approaches have been formulated in a wide range of fields from genetics to behavioural dynamics [7,13,25,27,30,33,35]. In the simplest setting of two strategies or traits, from the hopping rates T + , T − we havė 17) where ∇ = ∂/∂x, a(x) = T + − T − and b 2 (x) = (T + + T − )/N have to be derived for each microscopic update process and payoff matrix.

Stability and drift reversal
The phase space average discussed in the next section can be interpreted as an approximation of the time evolution of an observable H, here, written for the case of two strategies only, and, for an ensemble of arbitrary initial conditions, we approximate ρ(x) by a constant. (Formally, one should distinguish between a continuous and a discrete phase space average; within this paper -apart from constant prefactors -the functions for H and the rates T are continuous and differentiable.) In the remainder we focus on functions H which also are Lyapunov functions of an internal fixed point. Then a decrease of H can be interpreted as a motion towards the fixed point, and an increase of H is interpreted as an escape from the fixed point. Thus we can define (omitting the subscript x for the phase space average) and define "drift reversal" as the change of sign of ∆H . If a change of system parameters (N, w, payoffs), leads to a changing sign of ∆H , one observes the fixed point gradually lose its stochastic metastability. Due to this gradual transition, respective critical population sizes are approximative and could be defined in (possibly many) different ways. The essential advantage of assessing the stability from a sign reversal of this average drift ∆H is that it can be calculated comparatively easy, and in the case of the Battle of the Sexes, even analytically.

Average drift: Battle of the Sexes
For the replicator equation, Eq. (5) defines a constant of motion. As we are interested in the finite-size corrections, we can use H as an observable for the distance to the interior fixed point. For the processes defined above, the transition probabilities allow to calculate the average change of the constant of motion within the state space In the continuum limit the sums are replaced by integrals, For given transition probabilities T the drift of the discrete process can be expressed exactly, and in the case of linear processes, the resulting integrals are simple polynomial integrals. From this versatile expression, we can perform a comparison of the different processes.

Neutral evolution and Local Update
If w = 0, or if all payoff elements are zero, all terms of type (T +• − T −• ) vanish. Now consider the case where the payoffs come into play. For the Local Update, one has and Thus the term vanishes identically, i.e. for the average drift for the local update as well as for neutral evolution only the 1/N 2 order survives, The discrete stochastic diffusion in this neutral case is equivalent to the genetic drift of two independent types [36,37] and the 1/N term vanishes, as expected. But it is remarkable that the term in order 1/N does not contribute for the Local Update process. As this term corresponds for N → ∞ to the ordinary replicator equation, this is fully consistent with H being a constant of motion. For processes that depend on the payoff difference in a nonlinear manner, as the Fermi process, H is no longer a constant of motion. However, the behaviour of ∆H is remarkable, and investigated for finite populations in the next subsection.

General nonlinear pairwise comparison processes
Before approaching the nonlinear reproductive fitness defined by the frequency-dependent Moran process, it is instructive to analyze what happens if one considers a general pairwise comparison like for instance the Fermi process (14). Let us assume the general case that the difference between the reproductive functions ΦÃ − ΦB, which is antisymmetric upon interchanging the strategies A, B of the individuals anyhow, merely is a function of the difference of the payoffs which is fulfilled for the Local update, the Fermi process, and practically all common local comparison processes including "Imitate if Better". Then f is odd, f (x) = −f (−x). We also assume that this function is identical for the female and male population -this is justified as those differences should be cast into the payoff matrix rather than manipulating the reproductive functions. For convenience, one can here require f to be infinitely differentiable. Then f has only odd Taylor coefficients, The linear term is that of the Local update, whose cancellation in ∆H we have seen before. Upon inspection of, e.g., the order (2n+1) term, one finds apart from common factors Hf (2n+1) /(2n + 1)! that the integrand contains That is, for every n, the integral in order 1/N splits into the difference between two integrals which are symmetric in interchange of x and y, thus identical, and the contributions of each Taylor coefficient cancel. Hence follows Theorem 1 For a pairwise-type comparison process, where the difference between reproductive fitnesses Φ A −Φ B is an infinitely differentiable function of the payoff difference, the average drift ∆H in the Battle of the Sexes (with the ±1 payoff matrix) is independent of the strength of selection w and equal to the drift of neutral evolution (25).
As a corollary, there is no drift reversal for the Local update as well as for the Fermi process, confirming the numerical results in [25,29].
The result is remarkable in one sense. Even though a nonlinear comparison process can lead to a nonlinear replicator equation -which e.g. takes the form of a tangens hyperbolicus of the payoff difference for the Fermi process [32] -the average drift remains unaffected even for strong selection and small populations.

Linearized Moran process
For the linearized Moran process, one has for the male population a corresponding equation holds. The integrand in the diffusion (order 1/N 2 ) term, apart from a factor H, is As the sum of the payoffs A and B is zero for both populations, and again the game does not contribute to the diffusion term, which is, independently of w, identical to the neutral case. Now, does the game contribute to the first order? Here, and thus again the term vanishes, i.e. for the average drift for the local update as well as for neutral evolution only the 1/N 2 order survives,

Moran process
For the Moran process, it is possible to proceed even despite the nonlinearities. The Moran payoffs are, e.g. for males in strategy A, Using π A | + π B | = 0, and π~ + π | = 0, it follows In the weak selection limit, w → 0, this term approximates the constant 2. For w → 1, it vanishes asymptotically, but is negative for 1 2 < w < 1, due to a pole at w = 1 2 . While for x = 1 2 or y = 1 2 (a cross of lines in the state space) the average fitness vanishes, in the corners the square of the average fitness always is 1. The difference terms in the reproductive functions are The 1/N term thus becomes, observing π~ = − π | , The average drift for the Moran process then reads For weak selection w ≪ 1 2 , and N ≫ 1, one can approximate the last fraction, yielding

Weak selection limit of the drift for the Moran process
Despite reproductive success is based on a comparison with the average payoff (instead of a pairwise comparison), for the linearized Moran process investigated in Section 5.3, the drift reversal is lost. Thus, performing the weak selection limit at the level of the microscopic interaction does not properly conserve the properties of the average drift of the frequency-dependent Moran process.
But from the exact theory for the Moran case, one can consider the approximation w → 0 at a later stage: Both terms cancel for a w that matches One should bear in mind that due to the approximation w ≪ 1/2 made above, equations (31) and (32) rely on the weak selection limit, which is the biologically most relevant case.
An interesting viewpoint on its own is to consider w as a "bifurcation" parameter. Conversely to a Hopf bifurcation, where an oscillation amplitude grows with the square root of a bifurcation parameter, here the effect of the Moran dynamics, to stabilize the finiteness-stochastic oscillations around the fixed point, grows quadratically with the strength of selection w. Thus, for the frequencydependent Moran process in finite populations, the onset of the fixed-point stabilizing mechanism is a quadratic effect of weak, but increasing, selection.

Discussion
The investigation of the different update mechanisms suggests the existence of two universality classes: one, for which the internal fixed point is neutrally stable (in the N → ∞ limit), and a second, for which the internal fixed point is asymptotically stable (in the N → ∞ limit).
The Local update process as well as other (differentiable and pair-symmetric) pairwise comparison processes, as the Fermi process, and also a linearized Moran process, belong to the first class. The Moran process belongs to the second class, and one may assume that more update processes (combining average fitness and a nonlinearity) could be constructed for this class; the necessary condition is that the internal fixed point is attractive in the N → ∞ limit.
In this sense, the properties that were known for the deterministic replicator equations were recovered, and for the Moran process class the transition between the instable and (meta-) stable regime can be described by calculating the drift reversal from ∆H . This approach may be advantageous in high-dimensional cases, where exact simulations of the process become costly; a phase space average of ∆H is obtained much easier, and in very high dimensional spaces one may approximate by a Monte Carlo integration.

Conclusions
Cyclic coevolution in biological or social dynamics is an interesting class of coevolutionary dynamics and allows in an examplaric way to study stability, drift and diffusion in finite populations. Population sizes in markets, decison processes, as well as in biological populations from bacteria to ants and to vertebrates can vary by many orders of magnitude, thus the influence of the population size by the stochasticity in finite populations is of fundamental relevance in all those disciplines.
For the simplest generic model of a cyclic coevolution in an asymmetric conflict game between two populations, the "Battle of the Sexes", analytical results for the average drift could be obtained that support the consistent understanding how the different fixed point stability of ordinary and adjusted replicator equations translate to finite populations.
Partially the results are quite unexpected: Not only for the Fermi process, but also for a quite general class of pairwise comparison processes -even when they lead to different replicator equations -the average drift is identical to the neutral case, and equals 1/(18N 2 ). The second surprise is that, for the average drift, the linearized Moran process also falls into the equivalence class of the neutral evolution.
But how does the drift reversal seen numerically (and known in the N → ∞ limit for the adjusted replicator equation) then have to be explained? Here one has to calculate the average drift for the Moran case, using the nonlinear dependence on the average payoff to observe the drift reversal. The hereby obtained average drift then can be analyzed for weak selection, resulting in an estimate of the critical population size which delimits the stability regimes of ordinary and adjusted replicator equations.
More generally speaking, one can conclude that, to stabilize the coexistence of strategies in a large -but finitepopulation, not only the global knowledge of the overall success of the population -the average payoff -must be taken into account, but in addition it has to determine the reproductive fitness in a nonlinear manner. The (on process level) linearized Moran process, as well as neutral evolution or pairwise comparison, leads to quicker extinction of all but one of the four strategy pairs.