{'fraction_non_alphanumeric': 0.035623862647361344, 'fraction_numerical': 0.010503204367434132, 'mean_word_length': 4.58085881443868, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 6, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 10, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Enabling Imitation-Based Cooperation in Dynamic Social Networks

The emergence of cooperation among self-interested agents has been a key concern of the multi-agent systems community for decades. With the increased importance of network-mediated interaction, researchers have shifted the attention on the impact of social networks and their dynamics in promoting or hindering cooperation, drawing various context-dependent conclusions. For example, some lines of research, theoretical and experimental, suggest the existence of a threshold effect in the ratio of timescales of network evolution, after which cooperation will emerge, whereas other lines dispute this, suggesting instead a Goldilocks zone. In this paper we provide an evolutionary game theory framework to understand coevolutionary processes from a bottom up perspective - in particular the emergence of a cooperator-core and defector-periphery - clarifying the impact of partner selection and imitation strategies in promoting cooperative behaviour, without assuming underlying communication or reputation mechanisms. In doing so we provide a unifying framework to study imitation-based cooperation in dynamic social networks and show that disputes in the literature can in fact coexist in so far as the results stem from different equally valid assumptions.


Introduction
From social media, to power grid [33] and road systems [23], to mycorrhizae connecting trees [53], networks play a major role in promoting desirable behaviour. They can improve transport of nutrients or goods [59], they can connect long lost friends [64,60] and they can even make computer systems resilient to attack [20].
The multi-agent systems community has been greatly concerned with social networks and their effect on interaction since its very start. Social network analysis has been used to derive and extract systems of reputation [44,40], to explain the emergence of cooperation [14,45] or conventions [1] and to explore mechanisms of ostracism [38]. Networks have also been used as a policy/control tool to prevent polarisation [50], to promote cooperation via partner selection [49,48] while also being an emergent property out of reciprocity [36].
When the network is formed by self-interested agents engaging in strategic interaction, some desirable global properties, i.e., achieving Pareto-optimal outcomes, can be hindered by the desire to achieve higher payoff for themselves. This is particularly true in games such as the Prisoner's Dilemma, when the benefits of "cooperation" are outweighed by those of "defection", but the latter is the individually rational choice.
Cooperation may mean contributing to some collective-risk social dilemma (e.g. investments in green energy) to avoid a collective catastrophe to occur, while defection means no contribution at all [54]; in other settings cooperation may mean incurring some kind of cost (e.g. a tax) to receive social services and defecting means tax-avoidance while benefiting from said services [15].
How do social networks, where agents connect and interact with one another, affect their levels of cooperation? And what happens when these connections change in time? Some theoretical results [16] backed by empirical evidence [17] have suggested that networks have little effect on cooperation or contribution [57]. However, these findings have been obtained using static networks, while a positive effect on cooperation was found experimentally in dynamic [41,62] and temporal [25] networks with some analytic backing [47,32,37]. Even static networks were found to improve cooperation in experiments [42], simulations [46] and theory [31,37,12], though it seems to be the case that dynamic networks are far more amiable to cooperation than their static counterparts [27].
Despite these discrepancies, experimental results have broadly agreed that those who cooperate tend to be more popular than their misbehaving counters. When networks are dynamic and ties are at least partly endogenous, subjects rarely broke links with a cooperator [41] partially causing them to have higher degree [4]. In fact as cooperators attract preferential attachment [46,39,37], they emerge as "leaders" with high payoff [9]. Assortativity for mutually cooperative links arises out of subjects avoiding defectors when connections are formed bilaterally [62] and via unfriending misbehaving neighbours formed unilaterally [10,41].
Through ostracism [26] and punishment (for example sanctions as a form of costly ostracism [38]), the co-evolutionary process generates networks with scalefree degree distributions, that promote even more cooperation [46], and are heavily clustered [10] around cooperators [43]. Imagine, thus, a highly-interconnected subgraph or core of cooperators, collectively working and benefiting one another, surrounded by parasitic defectors in the periphery, mostly avoiding one another while clinging to cooperators in the core.
There is now solid theoretical and empirical evidence that reputation is an important mechanism for promoting cooperation [29,28,7,35,44]. Here we take an alternative approach and look at the emergence of cooperation when reputation or other communication mechanisms are not available or not reliable enough, and simply focus on the effects of imitation strategies and network dynamics on promoting cooperation. We see our results as complementing reputation research, by showing, among others, when cooperation cannot be sustained by imitation and partner selection alone.
One method to completely decouple reputation-like mechanisms from the topological process is to impose partnerships exogenously. In doing so the importance of time scales is highlighted; when edge activity is "bursty" -i.e. narrow, sudden spikes of activity -cooperation is impeded while intermediate temporality analytically maximises cooperation [25]. Although other studies found a similar Goldilocks zone -maximal gain for an intermediate value(s) of input(s) -in the time scales [52], much of the literature discusses a threshold effect or time scale separation. Some experiments have reported no evidence for such a threshold [62] while others point to its existence [41,22], backed by a slew of theoretic work [37,32,47] that have direct analogues in percolation theory [34,8]. The lack of evidence seen by [62] comes from considering too restricted a span of time scales, specifically the ratio between strategy and tie update rates, and thus requires further experiments to verify.
Such conflicted literature reflects the wide variety of often disjoint assumptions, frameworks and foundational concepts, at least in terms of temporal aspects. There is certainly a disconnect between theory and experiment; the former mostly considers large time evolution [37] or steady-state results [32] while experiments are typically limited to short term studies on the scale of tens of rounds [41,42,62]. There is then great variety in partner selection (e.g. random, round-robin, preferential) for experiments or dynamic tie updates for simulations, occurring in different regimes of the ratio between strategic and topological time scales [47,32,13]. Moreover, such experiments rarely, if ever, vary the ratio of timescales and as such there is very little empirical data regarding timescale separation. Understanding systematically why theories offer very different predictions, each backed by their own set of experiments, is a crucial task to truly understand cooperation. The core of such a task is to identify where fundamental assumptions align, differ or are compatible in a systematic way that has yet to be undertaken by current research.

Contribution
We propose an evolutionary game-theory framework, which call the Cooperative And Networked DYnamic (CANDY) framework, to disentangle the basic assumptions that enable cooperation in dynamic social networks, only relying on basic reality-resembling imitation strategies. CANDY starts with assumptions on agents' decision-making -how someone decides to cooperate/defect (the imitation strategy) and chooses to befriend/unfriend others (the network evolution) -and pro-duces the resulting average payoff and total number of cooperators. An illustration of the type of results CANDY can produce is given in Figure 1.
Methodologically, this framework allows for rigorously testing and comparing different assumptions, finding sets of assumptions which are compatible with empirical data and overcome much of the literature heterogeneity. For instance one may suspect the division in whether the ratio of timescales produces a threshold or Goldilocks effect to be due to the differences in assumptions theorists made or in exogenous update rules imposed upon test subjects. Our framework allows the coexistence of such incompatible results by rigorously scrutinising the underlying assumptions.
To illustrate the power of the CANDY framework, we recover the theoretical results [32] for when edges undergo a birth-death process and strategies follow a Moran [30,58] or Wright-Fisher [21] process. Furthermore, we consider other edge update models (such as cooperator-popularity) and behavioural models (such as conditional cooperation) that capture the assumptions of other research lines, in order to illustrate the qualitative differences that emerge.
Moreover, we provide a nuanced discussion on timescale separation by reproducing both the threshold effect as seen in [41,22,37,32,47] and a Goldilocks zone for defection; in many cases we suspect such effects are really artefacts of the finite number of rounds occurring. In recovering both phenomena we highlight how sensitive results are to both initial conditions and to the assumptions of the researcher. That is to say, by having slightly different but equally valid assumptions, the qualitative results can be significantly different.
Finally we illustrate how one assumption/observation -that cooperators are more popular -can lead to the emergence of the same core-periphery structure, despite instrumentally different update rules. Such core-periphery structures have been observed in experiments with human subjects [55] and other agent-based models [51,43], even when edges can only be broken, not formed for agents with multidimensional opinion spaces; there is an intuitive correspondence between our cooperators and defectors to homogeneous and adversarial agents respectively seen in [51].

Paper Structure
Section 2 presents the mathematical setup, followed by the introduction of the CANDY framework. Section 3 analyses the partner selection mechanics, while Section 4 looks at the imitation strategies. Section 5 provides the main results, showing the behaviour of the CANDY framework on key (random) graph models. We then move to the discussion of the findings and some key pointers for future research. In appendix we provide some basic preliminaries on variables, expectations and dynamical systems.

Theoretical Model
For a simple graph G = (V, E) of N nodes playing a repeated Prisoner's Dilemma game, denote the adjacency matrix as A = (a ij : i, j ∈ V ) and the strategies as a vector s = (s i : i ∈ V ), where the binary strategy s can either be 1 (cooperate) Fig. 1 The evolution of a dynamic network following the extreme popularity update model, wherein cooperators (C) are always befriended, defectors (D) are always unfriended and strategies are fixed. Colours represent the type of edge: CC edges are in green, CD edges in orange and DD edges in red. Agents are initially placed on a random graph generated by the Erdös-Rènyi model with probability p = 0.2 then allowed to update every timestep. As t → ∞ a core-periphery structure emerges where cooperators inhabit the core and defectors loosely hang on in the periphery. Note that so long as a non-zero number of cooperators C > 0 remain any initial condition, under any of the partner-update rules, will eventually stabilise to this type of core-periphery configuration with a core of size C.
or 0 (defect) 1 . A cooperators pays a cost c per neighbour, such that each of her neighbours gains a benefit b. A defector, on the other hand, pays nothing and nothing happens. This payoff structure was chosen to match the predominant games considered in the literature. From these N (N +1) local variables we can find global/aggregate variables that are of the most interest. Given A and s we can find the payoff vector π = (π i ) by defining a modified Laplacian using the payoff-structure (b, c) where K = diag(k i ) is the diagonal matrix of degrees. Note we will also be using k = (k i ) to refer to the vector of degrees. The payoff vector is then related to the strategy vector by a simple transformation π = −L s (2) of which the average payoffπ, can be simply found.
As with most of the evolutionary game theory literature the two main quantities of concern are the fraction or number of cooperators and the average payoff (i.e. payoff per capita). The latter is less often considered, despite the insight it provides into the actual network structure and thus helps to understand both processes occurring. For reference, therefore, we write below the equations for the total number of cooperators C and the average payoffπ in terms of local variables.
Rather than dealing with strictly discrete variables, we can instead move to the continuum by considering probabilities -here such probabilities are also identically the expectations of binary variables. Specifically we will considerã ij ≡ P(a ij = 1) ands i ≡ P(s i = 1) as the probabilities for edge (i, j) to exist and for node i to cooperate at time t, respectively. We then assume such probabilities evolve due to two independent processes -a vector field acting on the adjacency matrix, f g , and one acting on the strategy vector f s . In other words in the joint probability space [0, 1] N 2 × [0, 1] N where a point represents an entire state, this point moves due to the 'velocities' f g and f s .

CANDY Framework
In general we can write down N (N + 1) coupled differential equations (DEs) for both success probabilities in terms of our two vector fields f g = (f g ij : i, j ∈ V ) and strategic process f s = (f s i : i ∈ V ).
More often than not it is easier to construct transition rates rather than the full differential equation. Thus let us write f g ij and f s i in terms of transition rates: given (i, j) ∈ E the rate to form said edge g ij (0, 1); given (i, j) ∈ E the rate to break the edge g ij (1, 0); given s i = 0 the rate to cooperate h i (0, 1) and finally given s i = 1 the rate to defect h i (1, 0) 2 .
This provides the basis for the entire Cooperative And Networked DYnamics (CANDY) framework. So long as the update rules have a closed form, the above N (N + 1) equations fully specify the dynamics. CANDY allows for update rules that are time-dependent, parameterised and/or heterogeneous -suffice to say an incredibly broad range of possibilities. Moreover, by integrating the vector fields f g and f s , the flows are fully recovered Φ = (Φ g , Φ s ) allowing researchers to sidestep lengthy and computationally heavy agent-based simulations.
The evolution of C andπ are further gotten by differentiating Eqs. 4 and 5 to get DEs in terms of the generalised processes.
In specifying the local evolution due to f g and f s , researchers are able to clearly lay out their assumptions, numerically integrate and finally compare predictions. This generative method follows the same ideology as much of the agent-based modelling community, but with the added bonus of allowing for comparisons between qualitatively different hypotheses as it provides a standard framework to work within.
In general solving for the two global variables requires local knowledge and solutions may not be analytically tractable. However under certain local processes, F C and F π may be written only in terms of C andπ (and generally t), in which case the global behaviour reduces significantly down to a system of 2 coupled DEs, or even a single equation. When analytic solutions do not exist, numerical integration still provides a marked computational improvement on pure agentbased simulations.
In the next two sections we look at the two dimensions of the coevolutionary process, that of partner-updates and that of strategy-updates. We treat the two processes as independent, much like the equations of a fluid flow can be broken down into component parts.

Partner-Update Rules
In this section we consider a broad range of graph-theoretic models (GMs) of partner updates, in other words how edges change over time. We do so to illustrate the variety of empirical observations and assumptions of relationship building/breaking that are as valid as one another but that may produce entirely different dynamics. In particular we highlight three GMs -an extreme version of empirical observations that cooperators are always popular, an active linking model [32] and exogenously imposed networks as in [25].
In later sections we will combine such models with different behavioural models (see the next section) of how agents choose to cooperate or defect. In so doing we arrive at a diverse set of dynamics, some producing timescale separation while others showing regimes of mass cooperation and mass defection.

Graph-theoretic Model 1: Extreme Popularity
Following empirical results that cooperators are more popular [41,62,4], we take the extreme limit where only cooperators are befriended and defectors are entirely unfriended. That is at each time step (discrete round) a pair of nodes i, j = i ∈ V is randomly chosen. If the alter j is a cooperator, s j = 1, then the ego i will unilaterally form an edge with j if none previously existed. Otherwise if j is a defector, s j = 0, i will unilaterally break ties with j if the edge already existed. Note that by specifying at each time step we have inadvertently set a graphtheoretic timescale τ g ; in the units of time-steps τ g = 1 but one could also use units defined by the total number of edges possible τ g = N (N − 1)/2.
From this, we can specify the transition rate for an edge (i, j) to form g ij (0, 1) or break g ij (1, 0).
The factor of 1/2 is the probability to choose the active node as the ego, i.e. the one to unilaterally make or break a tie. The first term in Eq. 12 is from if i is a cooperator and j is chosen as ego, j always connects, otherwise if i is chosen as ego then i connects only to another cooperator; the second term is when i is a defector and is the ego, thus can connect to j if j is a cooperator. Similarly, for Eq. 14, the first term comes from when i is a cooperative ego disconnecting to a defector; the latter half is when i is a defector so either if both i, j are defectors there's two ways to break the tie or a cooperator j disconnects to i. Finally the evolution ofã ij can be found by summing the gain terms and loss terms.
As a sanity check when both i and j cooperate and are connected, f g = 0 so the edge is stable. Similarly when both are defectors and disconnected f g = 0 so the lack of an edge is stable too. Finally in general, as strategies are not fixed, to compensate for their probabilistic nature, we can simply replace s i withs i . The utility of having such a kernel is that we can obtain an ODE for k i -simply sum over all j = i,k i = j =i f g ij -and by extension an ODE for the payoff per capita when strategies are fixed.

Graph-theoretic Model 2: Active Linking
We consider the active linking model [32] to illustrate how their assumptions lead to a bottom-up or micro-scale model that then lead to the same meso-scale and macro-scale results. In active linking, cooperators and defectors form edges at some constant rate α C and α B and that different edge types decay away at rates β CC , β CD and β DD for CC, CD and DD edges respectively. Notice unlike our first model, this assumes an agent is not biased towards befriending cooperators over defectors, counter to empirical results. Instead your strategy impacts how strongly you connect to anyone else; in a sense this views relationships as stemming from the individual and not from reputations/biases.
Similarly to extreme popularity we can arrive at the transition rates by considering when i is a cooperator then j is either cooperator or defector and vice-versa. Notice now instead of a single graph-theoretic timescale there are potentially several timescales (edge formation versus edge destruction), but where they are not significantly different one could simply take an average.
Substituting the above transition rates into Eq. 8 we get an incredibly long and verbose equation for f g ij .
Importantly, each of the three terms above correspond to the three types of possible edges forming and breaking. The first term (s i s j [· · · ]) denotes evolution of CC-edges since s i s j = 1 iff both i and j are cooperators; similarly the other two denote evolution of the CD and DD-edges respectively.

Graph-theoretic Model 3: Exogenously Imposed Networks
Consider when the network is exogenously imposed -perhaps the structure is synthetically produced or reshuffled by the researcher, or as in [25] the network is empirical, such as contact-networks. In this case N 2 equations are eliminated and the only interesting dynamics occurs within the strategic space, as the graphtheoretic space has been fully specified.
For instance in the discrete case, let {t 0 , t 1 , · · · } ∈ R be a sequence of timestamps at which the adjacency matrix has changed so that during each interval t m ≤ t < t m+1 the network is fixed. The graph-theoretic timescale can be defined as a statistic on the set of interval periods, for example the mean or the minimum of such a set. When strategies update following an imitate-payoff type rule (see BM1 below) slower than the network gets updated, cooperation is promoted while when τ s < τ g mass defection occurs [25].

Strategy-Update Rules
In this section we focus on how agents decide to cooperate or defect, in particular we consider three different mechanisms for the imitation of neighbours. One, when imitation is purely payoff-dependent of a random successful neighbour, similar to [46]. Two, imitation happens due to social pressures that is presented with a crowd of opinions disagreeing with her, an agent will change her strategy, in other words conditional cooperation [11,5] which is also known as the voter model [19]. Finally, as a hybrid of the previous two, where both payoff and the will of the crowd matters, the pairwise comparison rule [37].
There are, of course, a multitude of other more complex decision making processes -for example moody conditional cooperation [18] and tag-based cooperation when agents have observable traits [56] -however to address every single possible rule would be never-ending. Hence for the sake of a focused scope and for readability we focus only on three.

Behavioural Model 1: Imitate-Payoff
Here we consider strategies are updated by pure imitation that are discretely payoff-dependent, that is imitation occurs only iff the proposed alter has a higher payoff. Every ζ time-steps, η existing edges are picked randomly 3 . One node per edge is then chosen randomly to be the ego i, who will imitate the strategy of their partner j iff π j > π i . Measuring in time-steps the strategic timescale is τ s = ζ/η, while relative to the timescale due to the graph-theoretic process τ s = τ g (ζ/η).

Behavioural Model 2: Conditional Cooperation
Alternatively to the payoff-dependent model, we can instead consider conditional cooperation (also known as a voter model) [5,19] where an agent cooperates if their neighbours also cooperate. This is frequency-dependent and payoff-independent, embodying the social pressures to imitate and change strategies (see for example [2,65,61]). Every ζ timesteps, η nodes are picked randomly. Such a node i, with strategy σ, will switch strategies to σ iff the fraction of their neighbourhood with strategy σ strictly exceeds a given threshold v σσ . This rule is thus parameterised by two thresholds v = (v cd , v dc ), where v cd is the for a cooperator to start defecting -that is the minimum fraction of a cooperator's neighbourhood that are defectors -and similarly for v dc . Since nodes are picked, rather than edges, the most reasonable time unit to consider would be in timesteps hence τ s = ζ/η, otherwise when compared to the graph-theoretic process τ s = N −1 2 ζ η τ g .

Probabilistic Version
Rather than having deterministic rules, we let the η nodes update stochastically, depending on the fraction of cooperators (or defectors) in their local neighbourhood. In this treatment, the probability for a defector to cooperate is the fraction of their neighbour who cooperate and vice versa.
Unfortunately due to the denominator, taking the expectation of Eq. 18 does not result in substituting the variables for their probabilities. Consider the first term as a function σ i : A × S → R acting on the adjacency matrix and the strategy vector. Denote, as usual, the expected values asÃ = (ã ij : i, j ∈ V ) and s = (s i : i ∈ V ). We can approximate σ i with a Taylor expansion.
Taking the expectation of the Taylor expansion, the first order derivative terms drop away while the higher order terms are now multiplied by variances and covariances; the variance can be explicitly found in terms of the Bernoulli parameters V ar(a ij ) =ã ij (1 −ã ij ). Any second order derivatives (or higher) with respect to strategies disappear as f i are linear in strategies. Finally assuming pointwise independence, covariance terms will vanish, thus leaving an approximate expression for f i in terms of only the expected adjacency matrix and strategy vector.
Noting that the second term and higher order terms are O(N −2 ) for non sparse graphs, a first order approximation of σ i is sufficient so we can in fact replace the variables in Eq. 18 with the continuous version.

Behavioural Model 3: Pairwise Comparison Rule
In Model 1 (imitate-payoff) an agent imitates precisely one neighbour based on the pairwise difference in payoffs, following a step-function. In reality someone may copy a friend even if their payoff was worse; for a start there might be some leniency where a friend might earn $10 more but I do not instantly side with him.
Moreover, although a single friend may earn more via their alternative strategy, there may be social pressures to stick to my current strategy. Model 2 takes the extreme and assumes the only thing that matters is the number of friends with a given strategy, so that if 80% of my neighbourhood are cooperators I'm 80% likely to cooperate.
The birth-death process of [37] takes a hybrid/intermediate stance relative to Models 1 and 2. It assumes that a) payoffs matter for imitation but not as rigidly as a step function -instead it should be smooth and continuous -and b) that social pressures also matter, so that the probability to flip strategies is given by the average probability to imitate an alternative strategy. Explicitly, the probability p ij for i to imitate the alternative strategy of j, i.e. s i = s j , follows a sigmoid with parameter β measuring the strength of peer pressure and payoffs acting as the energies. The probability for i to flip strategies is the mean of pairwise p ij over alternative-strategy neighbours. It is worth noting here that the instrumentalisation of this process would involve choosing a number of nodes rather than edges to update as such there will not be any factors of 2 to differentiate between ego or alter.
Note that as payoffs in p ij can be simplified by Eq. 2, π i = − k∈V L ik s k , we can rewrite the exponent in terms of only strategies and adjacency π j − π i = k∈V (bs k − c)(a jk − a ik ). The evolution of the cooperation probability is thus given below.
Once the network is dynamic, simply replace a ij withã ij for when adjacency is probabilistic.

Results
In this section we present several results and discussions, some theoretical and some computational, in order to highlight the advantages of the framework. We begin with the graph-theoretic models sans strategy -that is cooperators and defectors are fixed but they may change friendships -to illustrate how the network evolves and the structures that emerge. As both extreme popularity and active linking produce similar core-periphery mesostructures, we then combine extreme popularity -being the simplest model -with the three behavioural models: imitatepayoff, conditional cooperation and pairwise comparison. In doing so we identify cases timescales separate and cases where they do not. Figures 2-5 we simulate using N = 20 agents, 100 runs of a Prisoner's Dilemma with (b, c) = (100, 50) to be in line with the experiments of [41]. Moreover, our simulations begin with a variety of initial network conditions, that is networks produced by different random graph generators.

For our computational examples and illustrations in
-Erdös-Rènyi (ER) -the canonical random graph model where pairs of nodes are connected with probability p = 0.2.
-Barabási-Albert (BA) -a preferential attachment model that grows a network of N = 20 nodes by continuously adding nodes with m = 3 edges.
-Random assignment (rBA) -the initial cooperators are randomly assigned. -Highest assignment (hBA) -the initial cooperators are assigned to the nodes with highest degree.
-Cooperator Clique (CClique) -the C = 15 initial cooperators are completely connected to one another as a clique, while the remaining defectors are entirely disconnected from all others.
-Complete -all nodes are attached to one another.
-Stochastic Block Model (SBM) -specifically we use the assortative planted partition model [6], a special case of an SBM, with 2 communities, with an ingroup edge probability of p = 0.8 and out-group edge probability of q = 0.02.

Fixed Strategy
By fixing strategies we can more clearly understand what graphs form as a consequence of f g . In particular we see the emergence of mesoscale structures such as core-periphery (specifically the continuous core-periphery model [3]). The strength of such structures depend on how friendly and attractive cooperator/defectors are as friends as well as the perseverance of friendship types, however in most realistic settings (where cooperators are universally popular and defectors are avoided) a cooperator-core forms while a defector-periphery struggles to attach themselves to the core while largely avoiding one another. Henceforth for brevity we denote the mesoscale structure of cooperator-core-defector-periphery as CCDP.

Extreme Popularity
For the extreme popularity graph-theoretic model, Eq. 16 can be analytically integrated, thus leading to equations forã ij , k i andπ.
In order to supplement the theoretical prediction (Eq. 23) we run agent-based simulations following the extreme popularity model. As we see in Fig. 2 regardless of the initial condition -that is the network generation model used to initialise friendships -all converge to the same payoff per capita (Eq. 24). Moreover, only a small subset of possible graphs can generate this valueπ * ; for X, Y the number of CC and CD edges respectively, 4X + 2Y = (N + C − 2)C.
In fact the graphs that form all exhibit core-periphery structure similar to what illustrated in Fig. 1, regardless of initial condition and partner-update rule, so long as they follow the assumption that "cooperators are popular" and that there are C > 0 cooperators -as we will see in the next subsection. In other words this cooperator-core defector-periphery (CCDP) is a stable configuration, along with an entirely disconnected networks of pure defectors.

Active Linking
In the notation of the original paper [32] denote the number of CC, CD and DDedges as X, Y and Z respectively; in terms of elements of the adjacency matrix, the edge-set sizes are simply the double sum over i, j > i of the elements a ij multiplied by the relevant indicator function. In other words by taking the sum of each of the three terms in Eq. 17 we recover exactly the evolutionary equations for X, Y and Z laid out originally [32], where X m , Y m and Z m are the maximum sizes of the edge sets given a number of cooperators C in the population. The steady state solution is thus given by In a realistic setting cooperators rarely broke from each other [41] so that β CC ≈ 0 while defectors are regularly unfriended by everyone [10] β DD α 2 D . We see therefore that X * ≈ X m , Z * Z m and Y * ∈ [0, Y m ]. In other words we can see a very clear core-periphery emerging, whereby cooperators form a core and defectors are typically isolated in the periphery. This mesoscale structure has been seen elsewhere in the literature, such as in multidimensional opinion spaces with only edge-breaking [51].
Once again we emphasise the emergence of a core-periphery, despite the difference in partner-update rules. In fact once strategies are allowed to change, as we will see in the next section, the two absorbing fixed points of the system are in effect core-peripheries of C = 0 and C = N . That is, either the system turns into  Fig. 3 Extreme popularity, imitate-payoff. After agents have played 5T rounds of a Prisoner's Dilemma, following GM1 and BM1, the final payoff per capita is plotted for different ratios of strategic to graph-theoretic timescales τs/τg. Each point is an ensemble average over 100 simulations, with colour denoting the initial condition; the black dashed line is a baseline if all agents cooperated and were on a complete graph. Note towards the fixed strategy limit the rate of cooperation is bounded-from-above by the initial level of C = 15 a complete graph of only cooperators, or an empty graph of only defectors; the stability of these two states is governed entirely by what strategy-update rules are implemented, i.e. the assumptions of how people decide to cooperate or defect.

Coevolutionary Process
Here we address the cases when both edges and strategies can update, and do so on timescales of τ g and τ s respectively. By varying the ratio of timescales, τ s /τ g , we are able to explore the bifurcations that occur and regard the threshold effect that emerges.
In particular we focus on combinations of extreme popularity (GM1) rule with the three behavioural models to produce very different timescale curves (Fig. 3-5), by simulating 100 simulations for each combination at different ratios of strategic to graph-theoretic timescales τ s /τ g . After 5T rounds the ensemble-average payoff per capita is measured and plotted. A commonality to note is towards the fixed strategy limit, the rate of cooperation tends towards the initial C = 15 precisely because exceedingly few people have changed from cooperate to defect (or vice versa).
For comparison, in experiments where a fraction k of subject pairs -such as in [62] and [55] -are picked at random to update every round, τ s /τ g = k. In much of the experimental literature [41,42,55], there are 3 typical values for k: the fixed (k = 0%), viscous (k = 10%) and fluid (k = 30%) conditions. They all found that the fluid condition has higher levels of cooperation than either the fixed or viscous cases, in other words higher k have higher cooperative levels. We replicate these results in with simulations across the different update rules (see Figures 3 and 4) below and show how other unseen phenomena may also occur.  Fig. 4 Extreme popularity, conditional cooperation. After 5T rounds of a Prisoner's Dilemma, following GM1 and BM2, the final payoff per capita is plotted for different ratios of strategic to graph-theoretic timescales τs/τg. Each point is an ensemble average over 100 simulations, with colour denoting the initial condition; the black dashed line is a baseline if all agents cooperated and were on a complete graph. Note towards the fixed strategy limit the rate of cooperation is bounded-from-above by the initial level of C = 15. Cooperators find it easy to defect in the left panel, requiring only 20% of neighbours are defectors. In the right panel a switch to defection requires a majority of neighbours to already be defectors

Imitate-Payoff
As seen in Fig. 3, there is a separation of time scales in most realistic graphs under extreme popularity (GM1) and imitate-payoff (BM1). After such a long period of time most of the realistic graph models behave near identically with mass defection occurring consistently for τ s /τ g < 10 and non-zero levels of cooperation subsiding above this threshold. The difference in behaviour between the static network limit and the dynamic network -that the fraction of cooperators is higher in the dynamic case -has been observed in experiments [62].
However for the CClique initial graph -a highly artificial/pathological network where all cooperators begin as friends with all defectors entirely disconnected -playing under the same rules a reverse Goldilocks zone appears. That is for intermediate values of τ s /τ g defection is rife, while towards the fixed network limit cooperation is nearly maximal.
From an individual's perspective placed in a realistic graph, defection is the optimal and preferential strategy -the population is mixed enough that defectors can infect cooperators. As such when strategies can update quickly enough, all players have enough time to defect hence mass defection occurs. However when strategies are slow to update, there simply is not enough time for everyone to defect; consider when τ s /τ g = 10, although all pairs have been updated around 5 times each, only around half of possible imitation updates have occurred.
In contrast, the CClique condition actually promotes cooperation at the individual level, at least in the early stages, precisely due to the core being resilient against defection. Note that the strategy vector s will only start to change once a defector has attached to the core -otherwise there will be no alternative strategy to imitate from. Provided the core is large enough, when strategies update rapidly the likelihood of a defector imitating a cooperator is far higher than the reverse, in other words the core converts defectors quicker than they can infiltrate the core. As the edges update quicker, more and more defectors can attach themselves to the core quickly enough to start converting the cooperators. After some point strategies become too slow for everyone to defect hence the payoff per capita rises again in the limit of fixed strategy. These two competing factors thus produce a reverse Goldilocks zone, where cooperation is minimised, not maximised, at intermediate ratios.
Finally note that for all realistic initial conditions when the network is static (τ s /τ g = 0), we reproduce the qualitative result, seen empirically in [16,17] and theoretically in [57], that static networks do not promote cooperation. In fact this behaviour appears again for different strategy update rules, such as the conditional cooperation in the left of Fig. 4. However let us be clear that this is not always the case, across all static networks and across all update rules; we can qualitatively capture the discrepancies seen in the literature.

Conditional Cooperation
As shown by Fig. 4 the choice of behavioural model, which a priori is as reasonable as any other, may produce vastly different results. For v = (0.2, 0.8), that is when cooperators easily defect, we see a similar curve for most realistic initial conditions as in the imitate-payoff case -heavy defection for low ratios with higher cooperation at higher ratios. However we already see differences, for a start when cooperators have the highest degree in a scale-free graph non-zero cooperative levels are maintained at all τ s /τ g . This directly contrasts the rBA condition, where cooperators are randomly assigned in a scale-free graph. This result alone implies possible policy implications in order to sustain global cooperation.
Second, unlike in Fig. 3, the CClique condition no longer produces a minimum in cooperators, instead there is a clear separation where mass cooperation occurs. If the simulations were to run longer then we would see mass cooperation at all ratios, whereas for the realistic initial graphs mass defection would occur. This suggests that for this parameter set, this rule combined with the network structure heavily promotes cooperation at all times.
Third, we see how the parameter values control the extent to which cooperation is promoted, looking at the right of Fig. 4. This happens because the threshold to defect is now much higher, so that cooperators remain cooperative, while defectors begin to cooperate, thus cooperative rate only ever increases.

Pairwise Comparison
For β = 1 the timescale curve of the pairwise comparison model, Fig. 5, behaves similarly to the conditional cooperation for high defecting threshold. That is cooperation is promoted across all τ s /τ g likely as there are already many cooperators to begin with so that a defector feels an immense amount of peer pressure to cooperate. In this case when β is lower, the resultant level of cooperation will be similarly smaller, as there is less pressure to imitate them; otherwise increasing β would increase the speed at which mass cooperation happens. Moreover, we suspect the role of initial number of cooperators to be vital, that varying C 0 ≡ C(0) will lead to a bifurcation in C ∞ ≡ C(t → ∞). Already two bifurcation points are trivial, C 0 = 0 and C 0 = N , as in both cases there are no alternative views to copy hence C ∞ = C 0 .
Interestingly here, the CClique initial condition produces the lowest payoff per capita -unlike under different behavioural models. Even nearer the fixed strategy limit, the payoff per capita is around 50 units short of the realistic networksthis suggests that inside the core roughly one cooperator has had the chance and decided to defect.
Consider the CClique's evolution at intermediate times -that is when defectors have connected to the core but no agent has changed strategies -where there are C 0 cooperators and each cooperator has on average d defecting friends (hence each defector has on average C 0 d/D cooperative partners. For a cooperator i and a defector j, the expected payoffs are π i = 50(C 0 −1−d) and π j = 100C 0 d/(N −C 0 ), such that the probability for i to imitate j following Eq. 19 and given the conditions of our experiment, is as below.
Here we see immediately that it becomes increasingly likely a cooperator i will defect, once d ≥ 2 defectors have connected to her. Moreover, when τ s /τ g >> 1, we can use Eq. 21 to estimate d(t) ≈ (1 − exp(−t/τ g ))D/2 -so that d grows rapidly past d = 2 towards d = D/2. In other words cooperators will be under immense pressure to defect. However as τ s is large, only very few agents have the opportunity to update their strategies, hence we typically only see 1 cooperator defect.  5 Extreme popularity, pairwise comparison. After agents have played 5T rounds of a Prisoner's Dilemma, following GM1 and BM3, the final payoff per capita is plotted for different ratios of strategic to graph-theoretic timescales τs/τg. Each point is an ensemble average over 100 simulations, with colour denoting the initial condition; the black dashed line is a baseline if all agents cooperated and were on a complete graph. Note towards the fixed strategy limit the rate of cooperation is bounded-from-above by the initial level of C = 15 mass cooperation, suggesting that discrepancies in the literature arise from different assumptions and experimental design than being necessarily descriptive of real-world behaviour.
Through our framework we have been able to reconstruct the qualitative results of many previous works, even where they may at first glance seem contradictory. For a start, multiple experiments [41,42], agent-based simulations [46] and theory [31,37,12] show that non-zero levels of cooperation are maintained when strategies are updated sufficiently quick (τ s /τ g small) or similarly when networks are fully static; we see this across all initial networks for the pairwise comparison rule (Fig. 5) and for the conditional cooperation under the right parameter regime (right of Fig. 4). Moreover, for some rules we considered, we see a threshold effect in the timescales that has been theorised by [32] and [47] while in other cases we see a distinct lack of one as in the conditional cooperation (right of Fig. 4) reported by [62]. Finally we are able to replicate some of the empirical trends seen in experiments [41,62], that for higher τ s /τ g a higher rate of cooperation is maintained.
Our analysis suggests that although the speed of interactions (both behavioural and relational) is an important factor, it is not in general a sufficient condition for mass cooperation as suggested in [25], and is largely a by-product of the finite nature of the game. In the majority of conditions, we observed a threshold effect in the relative update speed τ s /τ g ; however given infinite time for any ratio of timescales, such a 'threshold' disappears, and mass cooperation/defection depends upon the condition. For example as seen in the conditional cooperation with extreme popularity, the initial network structure matters immensely and can either promote cooperation or promote defection. In other words, time-permitting, what truly matters is the context, the game structure(s) and the decision-making style of agents, not the relative speed of updates.
The one exception is in the single case of a so-called Goldilocks zone in defectors, where, for the pathological CClique initial network, a near static network promotes cooperation while rapid edge updates seem to favour defection. In the infinite time limit we would expect to see a threshold emerging.
Notable future research avenues include the study of games other than the Prisoner's Dilemma and the analysis of real-world group formation, such as working groups exhibiting hierarchical compartmentalisation and shifting pairwise interaction (e.g., Facebook or MS Teams).
Understanding the core-peripheral structures that emerge in real-world interaction, where cooperators collaborate closely with one another and defectors are ostracised, may have useful policy implications.
A further avenue for future work would be to explore a protocol whereby defectors are slowly engaged and introduced into the cooperator-core, to reduce the temptation on a cooperator to defect and to ease the defector into a more positive mindset.

Appendix A -From Variables to Expectations
As both a ij and s i are binary variables, we can instead consider them as independent 4 Bernoulli random variables, for all i, j ∈ V . In doing so, we can work in the continuous space of probabilities (specifically the parameter-space of Bernoulli distributions) which is equivalent to the expectations of our random variables, rather than the discrete space {0, 1} N (N +1) . By taking expectations, we can more easily analyse agent-based models by comparing the ensemble average with the statistical expectation.
As seen in Eqs. 4-5 we require distributions of Bernoulli variables, the sums, the products and the sum of products thereof. Fortunately by Proposition 1, assuming a ij and s j are independent, a ij s j is Bernoulli. The sum of such a product are thus Poisson Binomial by definition; Proposition 2 is a more general statement of this fact, the product of two Poisson binomial variables is itself Poisson binomial, in the terminology of Leemis [24] these distributions have the product property. This is vital as we have an analytic form for the expectation of a Poisson binomial variable Z ∼ P B(p 1 , · · · , p n ) given by E(Z) = n i=1 p i [63]. Proofs of both propositions can be found in [24].
Proposition 1 Let X and Y be two Bernoulli random variables X ∼ Bern(p) and Y ∼ Bern(q) with success probabilities p and q respectively. The product Z = XY is also Bernoulli Z ∼ Bern(pq) [24] with success probability pq.
Definition 1 (Poisson binomial distribution [63]) Let S n be the sum of n independent, possibly non-identical Bernoulli random variables X 1 , · · · , X n with success probabilities p 1 , · · · , p n respectively. S n is a Poisson binomial distribution, S n ∼ PB(p 1 , · · · , p n ).

Proposition 2
The product of two Poisson binomial distributions is Poisson binomial. Let Z X and Z Y be two Poisson binomial random variables, formed from n independent non-identical Bernoulli random variables {X 1 , · · · , X n } and {Y 1 , · · · , Y n } respectively, with success probabilities p X = (p X i ) n i=1 and p Y = (p Y i ) n i=1 . The product Z = Z X Z Y = i,j X i Y j is Poisson binomial Z ∼ P B(p X , p Y ).
As C is the sum of N independent non-identical Bernoulli distributions it is Poisson binomial, C ∼ P B(s), with mean i∈Vs i [63]. Moreover, by Proposition 2, the per capita payoff is similarly Poisson binomial, with mean i,j∈Vã ijsj (b − c)/N . In other words we can replace the local variables on the right hand side of Eq. 4 and 5 with their respective probabilities to find the expectations of the global variables, which for brevity we also denote as C andπ.

Appendix B -Dynamical Systems
For a differential equationẋ = F (x, t), defined by the vector field F , on a space X the solution is uniquely the flow Φ : X × R → X that maps a point x ∈ X to its position a time t ∈ R later. Definition 2 (Flow) A flow Φ on a space X is a continuous mapping Φ : X ×R → X such that for all x ∈ X and for all s, t ∈ R: Φ(x, 0) = 0 Φ(Φ(x, t), s) = Φ(x, t + s) LetÃ = [0, 1] N ×N be the space of N ×N expected adjacency matrices such that the ij th -element is the probability an edge exists between i and j. Equivalently one can think of this as the space of N × N adjacency matrices with bounded weights. Similarly letS = [0, 1] N be the space of N probabilities to cooperate, or equivalently the space of N continuous strategies. Conceptually we are thus considering a point (Ã, s) ∈Ã ×S in the product space, that moves continuously due to the vector field F , which represents the coevolutionary process.
Analogously to flows and vector fields in R 2 which have an x-component and a y-component, our velocity F has an 'adjacency-component', f g , and a 'strategycomponent' f s ; each 'component' is in fact either matrix-valued or vector-valued. Assuming the process does not explicitly depend on node-labels we can further assign two a time-scale to each component: a graph-theoretic τ g and a strategic τ s .