{'fraction_non_alphanumeric': 0.032038085434894496, 'fraction_numerical': 0.008663578658431977, 'mean_word_length': 4.50838648712497, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 10, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Non-Payoff Monotonic Dynamics in an Evolutionary Game of Courtship

We propose an evolutionary coordination game to formalize a simplified model of the evolution of strategies during human courtship. The dynamics, derived from the consideration of experimental observations on human social behavior driven by self-esteem, turns out to be non-payoff monotonic. This property gives rise to nontrivial evolution in the players' strategies, which we study both numerically and analytically.


INTRODUCTION
Nature offers a plethora of instances showing that relations between sexes are not always smooth. Extreme examples are the praying mantis female that cannibalizes its mate; the male bedbug, that slashes the female's abdomen with a sharp sting; and the toxin-loaded scorpion, that inoculates a special venom to subdue the female of the species. Aside from these drastic cases, most species show conflicting behavioral patterns between males and females when involved in a mating relation. In general, male behavior has evolved to crave scoring at any costs. On the contrary, the female is inclined to ensure viable offspring. This situation leads to a conflict between sexes concerning their respective shares in parental investment. Offspring raising requires a considerable amount of time and energy, that each parent might attempt to minimize at the expense of the other.
Humans, for example, require intense cooperation to raise children to adulthood [1]. They have evolved as cooperative breeders, and the presence of a helper is often essential to be able to raise offspring. But the conflicts that arise from breeding are not unique to humans. Whenever fertilization is internal [2], females risk being deserted even before giving birth to their offspring. Furthermore, as females produce relatively few, large gametes, whereas males produce many small gametes, the former are much more committed to breeding while the later are in a better position to desert. Males can invest time and energy not used in breeding into increasing their offspring with numerous mates. Male strategy is thus a fast and short engagement with the female. Females need adopting a counter-strategy to avoid desertion, and they insist upon a long engagement period before copulation. The complexity of the situation is interesting enough to attempt mathematical modeling. Game theory provides a suitable frame to discuss the so-called Battle of the Sexes [3] as a strategic game. Formulating a 2 × 2 game (two players, with two strategies each) where the male strategies are philandering and faithful, whereas the female strategies are coy and fast, it was shown that in a population of coy females, males must be faithful [2]. Among faithful males, however, it would not pay a female to be coy. This is an example of a coordination game, a class of games that we discuss in Section .
In most vertebrate species, the engagement of a male and a female is preceded by courtship. The courtship process in humans has been analyzed by psychologist in several studies [4]. In particular, in the first stage of courtship both genders have equal responsibilities in the successful evolution of the process. During their initial approach, the man and the woman make decisions based on parameters defined by their social conception and personality, which affect the subsequent course of the interaction. It was experimentally observed that, in general, women look for obtaining a stable relationship, unlike men who try to seduce the female seeking a sexual partner [5]. These dominant behaviors might have been socially shaped, and seem to be rooted on evolutionary paths.
In this paper, we analyze a simplified version of this problem. We consider two interacting populations: males and females. While individuals of both sexes seek to engage in a relationship with individuals of the opposite sex, their expectations are different. Males look for a direct approach to females, to get involved as soon as possible in a relation. Females, on the other hand, look for a more cautious approach, let seduction flourish, test their potential partners. We consider that both individuals will profit from establishing a relationship, that will prosper only if both partners match their respective expectations or strategies. A male will abandon his direct intent if the female is not immediately responsive, and a cautious female will generally not accept the approach of a man with no further intentions than sexual intercourse. After revisiting the concept of coordination game in Section , a mathematical formalization of the problem will be presented in Section .
The choice of a courtship strategy is expected to require highly adaptive decision-making abilities [6]. Among the constraints that affect this choice, we focus on self-esteem, which can be defined as the set of thoughts, perceptions, feelings about ourselves, built up out of the collected experience we grasp in every social interaction along our lives [7]. This self-evaluation directly affects the way how we relate to others and how we behave in social environments. People with different levels of selfconfidence, respond differently when facing the same social challenges [8,9]. Self-esteem, moreover, increases when an individual enlarges his/her record of successful interactions. In the case of courtship, this fact promotes the direct approach strategy among men and the opposite strategy among women, who become more selective when choosing their partners. In other words, the more successful the players are the more inclined they are to play their favorite strategy. In order to represent these aspects we introduce evolutionary game rules where the probability that an individual adopts either strategy depends on self-esteem. In turn, self-esteem is a function of the player's accumulated experience or score. Additionally, as a way of including social influence effects, we also consider the possibility of strategy imitation between males.

COORDINATION GAMES
A coordination game represents a situation in which players can make their strategies agree with each other in order to get the highest payoff. In a 2 × 2 game, suppose that player A can adopt strategies A1 and A2, and player B can adopt strategies B1 and B2. Coordination is possible when the payoffs, shown in Table I, satisfy A > B, ∆ > Γ, and α > γ, δ > β. In this case, the strategy combinations {A1, B1} and {A2, B2} are pure Nash equilibria [2]. Apart from these two pure Nash equilibria, and depending of the relative values of the payoffs, the coordination game may also have mixed Nash equilibria. It can be shown that the mixed strategy where player A chooses A1 with probability and A2 with probability 1 − µ, and player B chooses B1 with probability and B2 with probability 1 − ν, is also a Nash equilibrium. Unlike the pure Nash equilibria, however, the mixed equilibrium is not an evolutionarily stable strategy [10]. This can be proven by considering the replicator equations of the game [2], which state that the probabilities µ and ν evolve according tȯ , and φ B2 , are the average payoffs of individual strategies, namely The Nash equilibria of the game are also fixed points of the replicator equations, while stable fixed points of the equations are Nash equilibria of the game. Equations (1) has five fixed points, four of them corresponding to all the possible combinations of pure strategies, and the fifth corresponding to the mixed strategy. Among them, only the pure Nash equilibria {A1, B1} and {A2, B2} are stable.
In the following, we consider a subclass of coordination games, generally known as the Battle of the Sexes [3]. In the traditional formulation of this game, two players -Alice and Bob, say-try to decide how to spend their holidays together. Alice prefers going to the beach, while Bob is interested in mountain hiking. Despite this disagreement, however, both would prefer to go to the same place rather than to different ones. The Battle of Sexes is defined by the payoff relations A = δ > α = ∆.

THE GAME OF COURTSHIP
As a special case of the Battle of Sexes, we study a game with two players, male and female, each of them with two available strategies, direct (D) and indirect (I).
In connection with the courtship process described in the Introduction, the two strategies are respectively related to more open and more reserved attitudes. The payoff matrix is shown in Table II. While the coordinated strategies {D,D} and {I,I}, which we associate with mutually successful courtship, are more convenient for the two players than the other two combinations, it is clear that the male prefers D and the female prefers I.
In this game, the replicator equations for the probabilities m D and f D that respectively males and females play D areṁ  As in Section , we can verify the existence of a mixed Nash equilibrium with probabilities m D = 2/3 and f D = 1/3, apart from the pure Nash equilibria {0, 0} and {1, 1}. In Fig. 1a, we show a flow plot on the (m D , f D )-plane corresponding to Eqs. (3).
As stated in the Introduction, however, we are interested in a different type of dynamics, inspired on the effects of self-esteem during courtship. We assume that each individual plays the preferred strategy (D for male and I for female) with more frequency as his/her selfesteem grows. Moreover, we propose that self-esteem is determined by the degree of success (i.e., the total payoff) in previous matches. The probability m D of a male playing D is associated to confidence, and should increase as his total payoff grows. Meanwhile, the probability f D of a female playing D is related to her predisposition to open courtship, and should thus decrease with her total payoff. Calling Φ M and Φ F the respective total payoffs of male and female, and taking into account that the probabilities m D and f D must be bounded to the interval [0, 1], we choose the sigmoidal functions Variations in m D and f D are thus governed by the evolution of Φ M,F which, in turn, is given by the payoff in each match. Specifically, we havė where φ mD and φ fD are the average male and female payoffs when playing D with probabilities m D and f D respectively. Putting all these expressions in terms of m D and f D , we finḋ These equations have ten fixed points, with five of them coinciding with those of Eqs. (3). Figure 1b shows the corresponding flow plot on the (m D , f D )-plane.
Remarkably, the Nash equilibria of the Battle of Sexes are no longer stable under Eqs. (7). This draws an important distinction between the replicator dynamics and the self-esteem dynamics introduced here. The replicator dynamics is a special case of a broad family of imitation dynamics called payoff monotonic [11]. The most general form of imitation dynamics is given by the equationṡ where x i is the probability of playing a given strategy i among a set of n strategies, and the functions g i satisfy The condition for this dynamical system to be payoff monotonic is for all i, j. This means that the probability of playing a given strategy increases more rapidly if the corresponding payoff is higher, at the expenses of the less successful ones. In a series of well-known theorems [2,11], it is proven that for payoff monotonic equations (i) Nash equilibria are fixed points, (ii) strict Nash equilibria are asymptotically stable, and (iii) fixed points that are stable are Nash equilibria. Since these statements are not true for Eqs. (7), we conclude that our courtship dynamics is non-payoff monotonic, even when it was derived from social behavioral considerations.

NUMERICAL RESULTS AND THE EFFECTS OF IMITATION
Equations (7) were formulated assuming that the total payoff of any single individual evolves according to the average payoff at each match. Therefore, they do not necessarily capture the effects of "microscopic" fluctuations. Strictly speaking, moreover, they describe the evolution of strategy probabilities for a single male-female couple, while we are interested at analyzing the collective social dynamics of a population where many individuals of the two sexes have repeated opportunities of engaging into the courtship process. These aspects can be explored by performing agent-based numerical simulations.
We consider different population sizes ranging from 200 to 2000, with an equal number of males and females. Initially, each male or female strategy is defined by a probability of playing the direct strategy D, m D and f D . Through Eqs. (4), these initial probabilities define the respective initial (preexistent) total payoffs. Here, we analyze several situations with localized and uniformly distributed initial strategies.
First, we study the evolution of a population with strategies concentrated in each of the four regions indicated in Fig. 1b. From any of these initial conditions, the surviving strategy profile corresponds to one of the stable fixed points of Eqs. (7). Starting from the conditions indicated as a, the system converges to (0, 1); starting from b, to (1, 1/2); and starting from c and d, to (1/2, 0). Figure 2b shows typical evolutions corresponding to single simulations with 2000 individuals. For these localized initial conditions, hence, numerical simulations agree with the expected behavior derived from the equations.
The situation is however different when we start from uniformly distributed probabilities. In this case, numerical simulations show that the male population splits into two approximately equal groups, one of them converging to m D = 0 and the other to m D = 1. The female population exhibits the same behavior with respect to f D . This result can be understood by noting that, due to the symmetry of the initial condition, the system behaves at all times as if each male and female sees the population of the opposite sex as adopting the average strategy, respectively, f D = 1/2 and m D = 1/2. In fact, replacing these values in each of Eqs. (7), we geṫ The corresponding flow plot, shown in Fig. 3a, agrees with the numerical observation. Figure 3b displays the evolution of individual strategies in a single realization.
In order to take into account the effect of social influence, in the form of imitation, we introduce the probability q that, at each match, the participating male chooses its strategy by adopting the probability of another, randomly chosen male, instead of his own. For q = 0, we reobtain the original model and, in fact, as q remains small enough the collective behavior is the same as before. In particular, from uniformly distributed probabilities, the populations of both sexes split into similar groups that asymptotically approach the extreme values. For the sake of comparison with other instances, we call this asymptotic behavior case A. As the imitation probability q becomes larger, however, other kinds of asymptotic behavior emerge. In some of the realizations, all the male population converges to m D = 0 while the female population still splits into two groups that converge to extreme probabilities (case B). Alternatively, all the males converge to m D = 1 and all the females to f D = 1/2 (case C). As q grows, the frequency of case A decreases monotonically, while the frequencies of cases B and C, which are statistically identical to each other, increase. Figure 4 shows the fraction of realizations that end up in each of the three cases, as a function of q. We have run 1000 simulations for each value of q. The inset shows the results for different population sizes, while the main plot shows a collapse of these results obtained from standard finite-size scaling. A welldefined transition at q 0 ≈ 0.87, with a critical exponent γ = 0.2, is apparent.
As expected, thus, sufficiently frequent imitation between males leads their whole population to a common pure strategy. From a uniformly distributed initial condition, the two extreme values m D = 0 and 1 are equally probable, suggesting that the asymptotic state is controlled by microscopic fluctuations in the initial distribution of strategies. It is interesting that these two asymptotic pure strategies have a mixed collective counterpart in the female population: for m D = 0, each female indistinctly adopts either f D = 0 or 1, while for m D = 1, all females choose their strategies with equal probability, f D = 1/2. This behavioral diversity, no doubt, is to be ascribed to the non-payoff monotonic nature of the courtship game.

ANALYTIC APPROACH
In the following, we introduce a mean field-like set of differential equations for the system studied numerically in the preceding section. Our aim is to provide an analytical approach to the results obtained when social influence is included, and more specifically to check the existence of a transition between two different regimes, as shown in Fig. 4. We have seen that, when q = 0, the mean values of the probabilities of playing direct for each sex are m D = f D = 1/2. These values are also valid for the case A, while cases B and C, we have found (m D , f D ) = (0, 1/2) and (1, 1/2), respectively.
We assume that the dynamical behavior of each individual can be understood as if each player is confronting a virtual opponent who adopts the mean strategy of the population. Thus, each male plays against a female adopting f D = f D , while each female plays against a male adopting m D = m D . Moreover, the tendency of males to be affected by the social environment is quantified by the parameter q that defines the probability of males of playing m D instead of their own strategy. With these assumptions, our mean field-like equations reaḋ (10) with fixed points at f D = 0, f D = 1/2 and f D = 1.
Next we evaluate the stability of the fixed points in the above equations. As throughout our simulations f D always converges to 1/2, we fix f D = 1/2 and analyze the consistency of the dynamical behavior of the equations with the numerical results. It turns out that m D = (1 − 2qm D )/2(1−q) is unstable for any value of q and m D . As for the stability of the other two equilibria, Fig. 5 shows three regions in the (q,m D ) plane, that we have named A, B and C as they are in correspondence with the cases described in the preceding section. In region A, both m D = 0 and m D = 1 are stable, while in B and C only one of the two equilibria are stable, respectively, m D = 0 and m D = 1. In regions B and C, moreover, m D = (1 − 2qm D )/2(1 − q) adopts values outside the interval [0, 1]. Consequently, for q < 1/2 the two extreme values of m D are stable. For q > 1/2, on the other hand, a departure of m D from 1/2 can lead the system to select only one of these two values. This analysis helps us understanding why the state A disappears for high values of q, and why the states B and C are not present when q is low.
As for the stability of the equilibria of Eq. (10), we find that when m D < 2/3 both f D = 0 and f D = 1 are stable fixed points, while f D = 1/2 is unstable. When m D > 2/3 the stability properties become inverted, and the only stable equilibrium is f D = 1/2. In case B, we have m D = 0 and f D = 1/2. Accordingly, the population of females splits in two groups adopting the extreme values 0 and 1, while all males have m D = 0. For q > 1/2, fluctuations may lead the whole male population to abandon the strategy m D = 1 and choose only m D = 0. Again, as m D < 2/3, f D = 0 and f D = 1 are stable. However, when q < 1/2 unless a specific initial condition is selected, the system will converge to state A.
For case C the analysis is analogous to case B. Now, we have m D = m D = 1 and f D = f D = 1/2. When q < 1/2, f D = 1/2 is unstable, and m D splits between 0 and 1. Again, when q > 1/2 the system can converge to m D = 1. Once m D > 2/3, the only stable equilibrium is f D = 1/2. Despite the fact that this analytical approach disregards "microscopic details of the numerical simulations, thus, it correctly describes the collective behavior of the two populations in the three cases, and for small and large values of the imitation probability. CONCLUSION We have presented a simplified model to mimic the courtship process in humans as a game. Inspired by empirical evidence, we have proposed a coordination game where both players go better off by choosing the same strategy, i.e. acting coordinately, but where at the same time there are some constraints. Even when the best option for both players is to act in a coordinated way, there is no coincidence as to which is the optimal individual strategy. Consequently, coordination involves some kind of concession from at least one of the players. This typical situation is represented in the Battle of the Sexes. This game has been extensively analyzed due to the interest rising from the fact that it has two Nash equilibria, reflecting rational choices by the players. Rational players would agree that the non-Nash outcomes are disadvantageous, and that the Nash equilibria, even though one is inferior to the other on each player's opinion, are the reasonable option. Neither player would want to produce a disagreement that would lead to a non-Nash outcome, even if this implies a conflict. In a population of players, this rationality is well described by the replicator dynamics.
In our case, instead, we were interested at including non-rational but still quantifiable motivations in our players. Thus, we proposed a different dynamics, which still favors the possibility that the players coordinate with each other in order to mate. After unsuccessful interactions each player will modify the strategy becoming indulgent toward the opposite sex, even at the cost of moving off from the favorite strategy. In the absence of imitation, the dynamics is myopic, as the collected information is only related to the personal history. The proposed dynamics turns out to be non-payoff monotonic. This fact induces a dynamical behaviour completely different from the one obtained from the replicator equations. We verified the emergence of different dynamical equilibria and the change on the stability of the previously existing ones. Indeed, one of the most interesting features is that the Nash equilibria are no longer stable under this new dynamics. For some initial conditions, in fact, the dynamics leads the populations towards the most unfavorable situation. Decisions adopted by the players make them end up in an undesirable situation, far from the ideal coordinated outcome.
Further inclusion of social influence, in the form of imitation between males, adds to the behaviour of the system another interesting feature. Tuning the intensity of the imitation tendency we observe a transition between qualitatively different asymptotic behavior. While the imitation tendency is low the populations of both sexes split into similar groups that asymptotically approach the extreme values. As the imitation intensity among males becomes larger , the whole male population is led to a common pure strategy. The choice of the final state is governed by microscopic fluctuations in the initial distribution of strategies. Coordination seems to be enhanced as the information is shared: while males adopt a unified strategy, females fail to achieve the corresponding coordinated strategy to maximize their collective payoff. This