{'fraction_non_alphanumeric': 0.0321741070321989, 'fraction_numerical': 0.007584958976774162, 'mean_word_length': 4.665496419042269, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 15, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
The Role of Multiplex Network Structure in Cooperation through Generalized Reciprocity

Recent studies suggest that the emergence of cooperative behavior can be explained by generalized reciprocity, a behavioral mechanism based on the principle of"help anyone if helped by someone". In multi-agent systems, the cooperative dynamics is largely determined by the network structure which dictates the interactions among neighboring agents. These interactions often exhibit multidimensional features, either as relationships of different types or temporal dynamics, both of which may be modeled as a `multiplex' network. Against this background, here we advance the research on cooperation models inspired by generalized reciprocity by considering a multidimensional networked society. Our results reveal that a multiplex network structure may enhance the role of generalized reciprocity in promoting cooperation, whereby some of the network dimensions act as a latent support for the others. As a result, generalized reciprocity forces the cooperative contributions of the individual agents to concentrate in the dimension which is most favorable for the existence of cooperation.


Introduction
Ever since the pioneering work of Axelrod [1] on the stability of direct reciprocity (tit-for-tat strategy) in a lattice structured iterated prisoner's dilemma, a lot of effort has been put into discovering how different reciprocal mechanisms for emergence of cooperation fare under various topological circumstances. In particular, in [31,34,43] the concept of network reciprocity was examined, and in [36] the conditions that lead to promotion of cooperation through indirect reciprocity in complex networks were explored. Other notable studies include the role of emotions [25,26] or punishment of defectors [2,24,37], dynamical link formation through indirect reciprocity [17], and even considering continuous action profiles [40,39].
Under all these mechanisms cooperation evolves as an inherent feature of the competitiveness between the interacting agents. Recent biological studies, however, suggest that cooperative behavior can also emerge and be sustained if it is based on generalized reciprocity, a rule based on the principle "help anyone if helped by someone" [47]. In [42,7,20] it was shown that cooperation may emerge under this mechanism as a consequence of the changes in the physiological state of the agents caused by their positive experience from previous interactions.
The first steps towards the development of a framework to study the role of a state-based generalized reciprocity update rule in networked societies were made in [48]. In this work, the authors developed a simple model for pairwise interactions where agents send cooperation requests to randomly chosen neighbors. The acceptance of the requests is stochastically determined by a sole variable called internal cooperative state which reflects the agents' current welfare. A distinctive characteristic of the model is that, in steady state, the simple decision rule promotes cooperation while, at the same time, prevents the agents being exploited by their respective network environment.
While this and similar models shed valuable insights on the role that network topology plays in promoting cooperation, most of them have so far addressed only interactions on networks that are of one "dimension", ignoring possible multidimensional phenomena, i.e. multiplex network structures. This is an obvious drawback since real-life networks often exhibit heterogeneous properties within the edge structure that are of fundamental value to the phenomena present in the system [22]. For instance, in social network analyses the patterning and interweaving of different types of relationships are needed to describe and characterize social structures [10,50]. In telecommunication networks, the physical edges are often "sliced" into multiple parts in order to support the requirement of different devices [44,29]. Even genetic and protein relations between organisms constructed in multiple ways are crucial for the analysis of their global interaction properties [45,14].
To this end, here we extend the model introduced in [48] to account for a multiplex network structure, with the aim to characterize the network cooperation dynamics under the assumption of a state-based behavioral mechanism rooted in generalized reciprocity 5 . In our model the dimensions act as platforms which facilitate transactions between active members. The activity of the agents is modeled by constraining their presence to one dimension per round, and by making them able to answer only to requests from that same dimension. This assumption is consistent with the random walk models on multiplex networks [15], and is justified in systems where the round duration is very short and/or when agents have limited interaction capacities. The resulting mecha-nism, while preventing exploitation of the individual agents, exhibits additional features that act as promoters of cooperation in a multiplex network structure. Specifically, by allowing for heterogeneous benefits and costs (i.e. different parameter values across different dimensions), we show that cooperation can survive in the observed dimension even if the cost exceeds the benefit, as long as there is another dimension which acts as a support (having benefit-to-cost ratio larger than one). This essential characteristic of the new model comes in contrast to one-dimensional networks where the benefit being larger than the cost is a prerequisite for cooperation. Moreover, by introducing simple dynamics for the probability that an agent is present in a certain dimension, we show that, under a behavioral model based on generalized reciprocity, the individual cooperative contributions effectively concentrate to the dimension where most of their cooperative neighbors are also present. Based on these observations, we discuss connections to reinforcement learning, in particular to the model of Roth and Erev [41] and extensions therein [12].
The rest of the paper is organized as follows. In Section 2 we revisit the concept of generalized reciprocity and discuss the specifics of our state-based behavioral model. In Section 3 we introduce the stochastic network interaction model, together with its deterministic counterpart. We also introduce more details about the state-based behavioral update. The exposition in Section 4 (Results) is organized in 3 subsections. In Section 4.1 we derive the conditions for the emergence and stability of cooperation with homogeneous parameters across the multiplex network dimensions. In Section 4.2 we continue by relaxing this assumption and numerically examine the model properties with heterogeneous, i.e. nonidentical parameters across the network dimensions. This is done under the assumption of a random, but predetermined selection of the interaction dimension (i.e. fixed, predetermined probability of presence in a certain network dimension). In Section 4.3 we analyse the cooperation dynamics under a modified rule for the dimension presence, according to which the agents are free to adapt the probability of presence in each of the network dimensions as a function of their payoff (state-dependent probability of presence). The analysis is performed across different types of random multiplex networks, with the aim to investigate the role of the network topology. In Section 4.4 we test the model on an empirical dataset that describes the relationships between households of 75 Indian Villages (multidimensional networks) [5]. The numerical simulations on this real-life example support the general conclusion that the multiplex network structure, combined with an adaptive dimension update rule, enhances network cooperation in the multi-agent scenario with behavioral update based on generalized reciprocity. Section 5 (Conclusions) summarizes our findings and discusses possible directions for future work.

Background
Generalized reciprocity's roots lie within the concept of indirect reciprocity, a rule described as "help someone who is helpful" [11,33,47]. In the literature it can also be found under the terms of "upstream indirect reciprocity" [11] and "upstream tit-for-tat" [32]. In biological systems indirect and generalized reciprocity are essentially two different ideas since the former requires advanced cognitive capabilities of the involved entities. In particular, with anonymous interactions, applying indirect reciprocity requires that each agent keeps tracks of the reputation of all potential co-interacting partners, whereas generalized reciprocity requires each agent to know only his own recent history. Due to the complexity, indirect reciprocity has only been documented in humans [49]. On the other hand, real-life behavior based on generalized reciprocity has not only been found to be present in humans [7,4], but has also been observed in many other organisms, including rats [42], monkeys [23] and dogs [18].
The theoretical models that have been developed for the purpose of explaining the innate mechanism behind generalized reciprocity may roughly be divided into three main groups, though with possible overlaps. The first group encompasses deterministic behavioral update rules where the agents base their decisions of whether to cooperate or not solely on the outcome of their last interaction [38,16]. The models within the second group address the scenario where generalized reciprocity emerges as a result of a random walk in which the altruistic act of one agent initiates a chain (sequence) of similar acts across the network. Nevertheless, as stated in [32,13], this mechanism by itself is not sufficient for the promotion of cooperation unless other mechanisms such as direct or network reciprocity are already in place. The models within the third group address a behavioral update rule according to which the individual levels of cooperation are adjusted on the basis of an internal state reflecting the agents' general well-being (i.e. fitness). This adjustment takes place over time as a result of interactions with other agents [6,48].
The specifics of our model are such that it may be categorized as being in the intersection between the first and the third group. In particular, it shares the state-based behavioral update with the models from the third group. On the other hand, by approximating the stochastic interactions by a deterministic model, as done in [38], our model is related with the models in the first group. Nevertheless, differently from the other deterministic approaches where the strategy choice was assumed to be binary (either fully cooperate or defect), our model can be placed in a continuous iterated Prisoner dilemma framework [40,39], with the note that the choice of the particular strategy (from the continuous set of possible strategies) in each round is determined by the agent's state (reflecting its accumulated payoff, i.e. well-being).
Compared to the classical (i.e. one-dimensional) random graph model, the multiplex network structure offers additional degrees of freedom which render the extension of the one-dimensional models nontrivial. While the evolution of cooperation in multiplex networks has been addressed in the context of other types of reciprocity [19,8], our interaction model and the state-based behavioral update yield interesting implications on the cooperative behavior in these networks. Similar to [48], the accent here is on the role of the network structure and the thereby related neighborhood importance index, here generalized to account for the different temporal interaction model due to the multiplex structure.

Network interaction model
We consider a population of N agents whose relations are modeled as a connected multiplex network, defined as the triplet G (N , E, L), where N (the set of nodes) corresponds to the set of agents, E ⊆ N × N is the set of edges that describes the relationships between pairs of agents, and L is the set of L properties that can be attributed to the edges and which define the dimensions of the network. Formally, a dimension can be defined as the graph G [l] N , E [l] in which E [l] is the subset of E having the property l ∈ L. Each dimension is given via an N × N adjacency matrix A [l] , where the ij-th entry A The interactions between the agents are modeled as follows: in each round t, each agent i: 1. randomly chooses a dimension l where it will be present in that round; 2. sends a cooperation request to a randomly (on uniform) chosen agent j from its neighborhood in the l-th dimension, j ∈ N Given this interaction model, the random payoff of agent i at round t may be characterized as In (1), v i (t), which itself is a random variable describing the probability that i is present in layer l in round t. The selected index from the neighborhood of i is a random variable uniformly distributed on the set N k is a Bernoulli random variable with parameter 1/d k (t) captures the random number of agents (neighbors of i in l) which send a cooperative request to i in dimension l during round t.

Deterministic approximation
We approximate the stochastic model (1) by a deterministic model in which the random variables are substituted with their respective expectations The term z (2) is defined as is the temporal extension of the neighborhood importance index discussed in [48], in dimension l. This quantity acts as a local centrality measure of an agent, with agent i being more "important" in the addressed dimension if it has many neighbors which are at the same time also present in that dimension, and the neighbors themselves have few neighbors. In our model of interactions, this agent would be called upon rather frequently in the studied dimension. The motivation to use the deterministic model (2) is that it captures the long-term behavior of the stochastic model, i.e. provides reliable approximation of its steady state behavior. Numerical simulations of the stochastic model in the long run suggest that the stochastic variables can indeed be approximated by their respective expectations, without affecting the long-term network behavior, thus justifying the approximation. In Fig. 1 we display an example for the deterministic interactions between two agents that are placed on a two-dimensional network.
We also define the aggregate time-dependent neighborhood importance index Z i (t) of agent i as As we will see in more detail in Section 4 (Results), the probability distribution of the quantity Z(t) across the agents crucially determines the global cooperative behavior in the network. This quantity, which is a form of aggregated centrality measure in the multiplex network setting, critically reflects the role of the network topology on the cooperation dynamics in our interaction model. We elaborate on this in more detail in Section 4 where we address multiplex networks generated from Erdos-Renyi (ER) and Barabasi-Albert (BA) random graph models, as well as a real-world example of a social network describing relationships between households in Indian villages [5].
With the above, we write (2) in a more compact (vector) form as where

Behavioral update rule
We study a synchronous update rule, based on the accumulated payoff of the agent i by round t, Y i (t) = Y i (t−1)+y i (t), with Y i (0) being the initial condition and y i (0) = 0. The cooperative state of i at round t + 1 is defined as where we assume that the function f : R → [0, 1] is increasing. A plausible choice which reflects real-world behavior is the logistic function where the parameters k and ω 0 define the steepness and the midpoint of the function.

Analytical properties of the model
Here we characterize the main properties of the model in steady state. Hereby, we distinguish between two types of results: 1) results with homogeneous parameters, defined as identical benefit and costs across network dimensions, b [l] = b and c [l] = c for all l ∈ L; and 2) results with heterogeneous parameters, i.e. the more general case when we allow for different values for the benefits and costs across network dimensions. We remark that the proofs for the properties follow directly from applying the framework presented in references [46,48], nevertheless for concreteness in the exposition we present them.
The following property holds in general.
1. Robustness to exploitation. In steady state, the agents may be attributed to two disjoint sets, W = {w ∈ N : y * w = 0} and S = {s ∈ N : y * s > 0}, based on the steady state payoff y * i . The agents in S, which we refer to as "strong agents", are characterized by p * i = 1, while the agents in W, called "weak" may take both values p * i = 1 and p * i < 1, depending on the network parameters. Hence, there are two sets of relations that have to be satisfied Note that the sets W, S, the steady state values p * i , i ∈ W and B [l] * j , i ∈ N , l ∈ L and the constants y * i , i ∈ S are unknown. Proof: The update rule (5) yields the following set of iterative equations for i = 1, . . . , N In equilibrium it has to be fulfilled for i = 1, . . . , N . By applying the inverse map we get The above requires y * It is easy to verify that if there exists i such that p * i = 0, then the same is true for all i ∈ N . Indeed, when p * i = 0, then from (2) and since l B [l] * i = 1, it must hold that either: 1) y * i > 0, or: 2) p * j = 0 for all j in the neighborhood of i . The condition 1 implies p * i = 1, which is a contradiction. The condition 2 yields p * i = 0 for all i ∈ N by repeating the same argument to the nodes in the neighborhood of i, until all agents are reached. We note that this case is also covered by the requirement Θ * i p * = 0, with the solution p * = 0. Hence, an equilibrium fulfills p * ∈ 0 ∪ (0 1] N and is thereby characterized by non-negative steady state payoffs y * i ≥ 0. The following properties hold only for a multiplex network with homogeneous parameters. We relax this assumption in the numerical analysis performed in the following sections. 2. Existence of cooperation. A necessary condition for existence of cooperators (agents with p * i > 0) is b/c ≥ 1. Proof: Note that the total network payoff can be written as It is easy to show that, b/c < 1 implies p * i = 0 for all i ∈ N . Indeed, if there exists i such that p * i > 0, then the total steady state network payoff is strictly negative, implying that there is some i for which y * i < 0 (contradiction). Hence, the necessity of b/c ≥ 1 for existence of cooperation.
3. Promotion of cooperation. When b/c > 1, we observe the steady state probabilities are strictly greater than 0, p * i > 0 for all i ∈ N . Proof: By contradiction. If there exists i such that p * i = 0 then, as already discussed, it must hold that p * i = 0, for all i ∈ N . This, however, would yield a total network payoff i y * i = 0, which contradicts (8). 4. Sufficient condition for existence of strong agents. When b/c > 1, there is always at least one strong agent in the network.
Proof: This follows from the observation that when b/c > 1 the right-handside of (8) is strictly greater than zero, which implies that there is at least one i for which y * i > 0 and p * i = 1. 5. Necessary condition for the existence of strong agents. A necessary condition for existence of strong agents, (agents with p * i = 1), is Z * i ≤ b/c. Proof: The proof follows directly by substituting p * i = 1 and the fact that where Z * max is the largest neighborhood importance index in the graph, Z * max = max i Z * i , is both necessary and sufficient for all agents to be strong.
Proof: We note that the proof that p * i = 1, ∀i ∈ N , implies b/c ≥ Z * max , follows directly from property 3. To prove the converse, we use contradiction. We first define p min = inf p * i , i ∈ N , and set b/c to be greater than one (since b/c > 1 is prerequisite for cooperative behavior). Now, let us assume that the converse is not true, that is b/c ≥ Z * i for all i, and there exists some i such that p * i < 1. Under this assumption, for all i ∈ W we would have: For all i satisfying b/c > Z * i , (9) holds with strict inequality, whereas those i ′ for which b/c = Z * i ′ must satisfy p * i ′ = p min . This, however, can hold if and only if the agents corresponding to these indices are only linked to each other in each dimension and have the same degree in them, i.e. form a connected component. In that case Z * max = 1 = b/c which contradicts the assumption b/c > 1. Hence, the converse must also be true, which concludes the proof.

The role of heterogeneous parameters
We continue the analysis by relaxing the assumption of homogeneous parameters, and consider the situation where each dimension l has its own benefit b [l] and cost c [l] . Since our goal is to examine the effect of heterogeneous parameters, we develop a null model in which the probability for dimension presence is uniform in every round.
For this case, we compare two different multiplex networks each composed of two dimensions (the results can be easily generalized to networks with more dimensions). In particular, the first multiplex network type represents a natural generalization of the Erdos-Renyi (ER) random graph, while the second is a multiplex version of the Barabasi-Albert (BA) preferential attachment graph. In an ER random graph an edge between two agents has a fixed probability of being present, independently of the other edges. As a consequence the degree follows a Poisson distribution. On the other hand, a BA graph is constructed by a dynamical process in which in each step a new agent is introduced, and this agent makes connections to other agents that are already in the graph with probability proportional to their degree, thus ending up with a power law degree distribution. More details about the algorithms used for generating these multiplex networks can be read in references [9,27,28].
In what follows, we will consider networks which consist of 100 agents, and where the average degree in each dimension is 8. Since the correlations between the edges in different dimensions should play a prominent role in determining the steady state level of cooperation we are going to study three different scenarios. In the ER graph, we will examine the possibility of overlapping edges, i.e. situations i) when there is no edge overlap in different dimensions, ii) when half of the edges overlap, and iii) when all edges overlap. In the BA graph we study the cases where the degree correlation ρ generated through preferential attachment is i) negatively correlated, ii) there is no correlation, and iii) is positively correlated between the dimensions. Formally we measure the correlation between the degrees in separate dimensions through the Pearson correlation coefficient, i.e In equation (10) d [l] denotes the average degree in dimension l.
The results are depicted in Fig. 2. Panels (a)-(b) respectively show the evolution of the fraction of strong agents for the multiplex network generated by the ER and BA multiplex networks while the benefit to cost ratio in the second dimension, b [2] /c [2] is varied, whereas the benefit to cost ratio in the first dimension is kept constant (b [1] /c [1] = 1.08). In general, we observe that the ER networks require much lower benefit to cost b [2] /c [2] ratios for displaying full cooperation than the BA networks. In addition, in the case of ER networks, it can be noticed that full edge overlap is better for the level of cooperation displayed when b [2] /c [2] is low (see the inset plot of Fig. 2 (a)), while no edge overlap promotes more cooperation for higher values of the benefit to cost ratio in the second dimension. Similarly, positive degree correlation in the BA networks leads to higher fraction of strong agents for small benefit to cost ratios(inset plot of Fig. 2 (a)), while negative degree correlation is better for supporting cooperation when b [2] /c [2] is high.
We point out that due to the uniform probability of dimension update there is a symmetric relationship between changes in the benefit to cost ratios in both dimensions and the fraction of strong agents, i.e. a change in b [l] /c [l] in one dimension has the same effect as a change in the other dimension. Any slight adjustment in the probabilities only changes the symmetry towards one of the dimensions, which means that the results should not be gradually altered. Hence, we can use the properties derived in the previous section as a starting point for the inspection of the results.
This reduces the analysis to studying the steady state distribution of the index Z. We note that, according to our interaction model, higher values of the index imply more frequent cooperation requests, and hence, lower incentives for cooperation. For this purpose in panels (c)-(d) of Fig. 2 we plot the typical probability density function (PDF) for the Z index for the same random multiplex networks where the dimension presence is given by a uniform probability by averaging the index across 1000 network realizations. There, in black, we also  Fig. 2: (a-b) Fraction of strong agents σ as a function of the benefit to cost ratio in the second dimension, b [2] /c [2] , for a sample of the random graphs that we study, while b [1] /c [1]  depict the PDF of Z for the one dimensional ER and BA networks 6 . For the ER graphs, it can be seen that the exclusion of overlapping edges effectively increases the mode of the distribution. Since the average level of cooperation at the point where cooperation begins to exist is determined by the left tail of the distribution, the networks with lower modes, and thus larger left tails, should be able to promote more cooperation. Contrastingly, the fatness of the right tail leads to higher thresholds for displaying full network cooperation. In a similar fashion, we notice that by decreasing the correlation between the degrees in the multiplex BA graph, the right tail of Z decreases, and therefore the lower threshold for full cooperation. Finally, in the inset plots of Fig. 2 (a)-(b) we observe that the inclusion of a second dimension leads to significantly lower threshold for existence of cooperation in the system (which in the one dimensional case is b/c > 1 [48,46]). This implies that the other dimension acts as a support for existence of cooperation even if the original dimension does not allow it. This is a result of the fact that the negative payoffs from the original dimension are compensated with positive payoffs from the supporting dimension. If at least one agent receives higher steady state payoff from the supporting dimension than the loss in the original, then cooperation will persist. This is an important implication to the emergence of cooperation in systems where all dimensions of the network can not be observed and the environment is not suited for cooperation, while the phenomena is still detected.

The role of dynamics in the dimension update rule
Predetermined presence is a plausible assumption for systems where the flow between dimensions is constrained and agents are not allowed to develop beliefs about which dimensions generate higher payoffs to them. A more realistic case would be to allow for dynamics in the probability that agent i is present in dimension l in round t. While this can be modeled by introducing Markov transition rates for moving from one dimension to another, or even adding memory rates to the movement based on the experience in the previous rounds, here we consider a simpler update rooted in the same generalized reciprocity rule that was used for the internal state update.
Concretely, we consider an update based on the accumulated payoff in the dimension, Y [l] with Y [l] i (0) being the initial condition and y [l] i (0) = 0. In our model the updated probability of presence of agent i in dimension l is given by the softmax function We remark that the described rule is similar in spirit to the famous Roth-Erev reinforcement learning algorithm for strategies in extensive form games [41]. In fact, based on (11) and (12), an analogy with more general multi-agent reinforcement learning models can be established [12,21]. The connection is provided by interpreting the act of presence of agent i in dimension l of the multiplex network as a selection of a strategy S l (from a set of L preselected strategies), where the strategy selection is applied with probability B [l] i . In this context, the total payoff Y [l] i in the network dimension l is analogous to the propensity to play strategy S l . We note that the here applied rule (12) is different from the one introduced in the original Roth-Erev learning model, according to which strategy l is selected with probability B [l] . Specifically, it can be considered as a special case of a more general reinforcement learning scheme in which the probabilities for players to choose certain actions are taken from a general Gibbs-Boltzmann distribution In (13) λ plays the role of "inverse temperature" in statistical physics, and captures the trade-off between exploitation (λ = ∞), i.e. greedy learning in which only the action with the highest propensity is taken, and exploration (λ = 0), meaning that all actions are equally probable. In many reinforcement learning problems, the key is to find a value of λ that achieves a reasonable tradeoff between exploitation and exploration in the model of question. In our model, the selection of this parameter would critically determine the behavior in the system when, for example, some dimensions are erased. We expect that in that case selecting the parameter λ towards the "exploration mode" would provide certain robustness to such events. The exact quantification of these effects under this scenario, together with the role of the network topology, is out of the scope of this manuscript. However, it represents an interesting direction for future work. The advantage of the suggested update is that it can be very easily implemented since the agent only needs to know the probability B  i (t) from them. We point out that the steady state solution of equation (12) is not defined if more than one Y [l] i tends to infinity. This never happens as long as each agent experiences different dynamics when the dimensions are considered as separate networks. Another thing worth emphasizing is that the resulting system has (L − 1) N degrees of freedom, and, hence, complex behavior is unavoidable. Therefore, in the comparative statics as starting points for the probability that an agent is present in a certain dimension we consider real numbers whose values are comparable to the steady state of the preceding benefit to cost ratios. In the beginning, at the lowest benefit to cost ratios, the starting point is set to be equal among all agents and dimensions.
The results for the same networks as in the previous section are shown in Fig. 3. Panels (a)-(b) depict the fraction of strong agents as a function of the benefit to cost ratios. On the one hand, we observe that the global level of cooperation displayed for low b [2] /c [2] ratios is increased by a large amount when compared to the predetermined probability for dimension presence. On the other hand, we notice that for larger benefit to cost ratios, the overall level of displayed cooperation is not consistent in terms of performance. More precisely, there are situations in which it is decreased (e.g. the no-overlap ER network), and there are situations in which it is increased (e.g. the positive correlation BA network) when compared to predetermined presence.  [2] /c [2] , for a sample of the random graphs that we study, while b [1] /c [1]  This aggregate behavior can be explained by looking at (c)-(d) of Fig. 3, where we display the fraction of individuals which in steady state are present in the first dimension as a function of the same parameters. Obviously, the dimension in which the agents are always present in steady state is not always the same, i.e. it is dispersed among the agents depending on the network topology and parameters. This is a key feature of the model since it implies that the dimension update rule forces the agents to accommodate their presence towards the dimension where they either carry the smallest burden to cooperate or where most of their cooperative neighbors are present. As such, when coupled with the generalized reciprocity state update rule (5), the dimension update rule facilitates the promotion of cooperation in the system (in the sense that it promotes the existence of agents with p * i > 0).
It is worth mentioning that the fact that agents may accommodate their presence to the dimension where most of their cooperative neighbors are present indicates that full unconditional cooperation is not guaranteed to be achieved in an easier fashion. In particular, the dimension update rule may force an agent in steady state to be present in a dimension which is less suited for its personal gain than some predetermined rule just because most of the other agents are present in that dimension. Exactly this may be the cause for the lower level of cooperation exhibited in the no-overlap ER network. Due to the manner in which the edges are constructed in this graph, it may happen that some agents to have high neighborhood importance index in the first dimension but low in the other. For these agents, a uniform predetermined rule would imply that they would be able to generate a higher payoff by interacting in the dimension where they are less burdened. However, none of their neighbors with lower value of Z would be better of if they are present in it due to lower benefit to cost ratio and, more importantly, due to being less forced to answer to a cooperation request. As a consequence, these group of agents accommodate their presence in the second dimension, and the agents with high neighborhood importance index in it are also forced to be present in it.

Real-world examples
As a means to provide an intuitive example for the experimental application of the model we utilize social network data that describes relationships between households in Indian Villages [5]. In this dataset there are a total of 75 villages (networks) each represented through 12 separate dimensions. Since, there is a significant overlap in the way the separate dimensions are constructed (see [5] for a detailed description), here we consider only 4 dimensions that describe essentially disparate types of social interactions. In the first dimension, the links represent relationships between households who help each other with making decision, the second and third, respectively, describe the borrowing interactions of money and kerosene and rice. Finally, the fourth dimension are the medical advice relations.
For estimation purposes, we exclude households that have no neighbors in at least one of the studied dimensions. Thus we end up studying 75 different multiplex networks each consisting of 4 dimensions and on average 126.45 individuals (with a standard deviation of 43.59). More detailed summary statistics are given in Table 1.
For easier interpretation of the results, in the numerical estimations we set homogeneous benefits and costs. In Fig. 4 panel (a) we plot the average fraction of strong agents as a function of the benefit and cost ratio b/c across all multiplex networks with and without the dimension update rule. There, we also depict the same variable when each dimension is considered as a separate network. In general, we observe that the dynamics of the examples in which the dimension update rule is at force supports most cooperation, followed by the uniform update rule. In this particular example, the individual dimensions behave as worst promoters of cooperation based on generalized reciprocity. Similar to the previous results, these are also best explained if we look at the steady state distribution of the index Z (panel (b) of Fig. 4). In particular, it can be seen that the average distribution for Z in the multiplex network with dimension update rule has tails that are less fat than the other example structures. Clearly, this is a result of the agents being able to accommodate to the dimension where they receive highest payoff, differently from the other structures that are compared. As evidenced in panel (c) of Fig. 4, where we show the average steady state probability for dimension presence (averaged across agents and estimations), in steady state the agent presence distribution for the dimensions is well diversified. While most of the agents choose to be present in the Medical advice dimension, there are some agents that favor the other dimensions. Consequently, the mass of the distribution of Z is driven towards one. In other words, Property 5. is easily satisfied for most agents solely by assuming b/c > 1 and Property 6. is reached with lower b/c ratio. On the other hand, all other distributions exhibit fatter tails, and hence require higher benefit to cost ratio in order for full cooperation to appear. This is especially true for the help with a decision dimension when it is considered as a separate network. Evidently, in it there is a small group of agents that receive cooperation requests way more often than they send. The fat tail of Z in the help with a decision dimension can be a direct result of the fact that this dimension has on average the lowest clustering coefficient. Concretely, many of the possible triads of edges are not formed, which in turn leads to excessive burden to a particular group of agents for which the edges in the possible triad exist.

Conclusions
The emergence of cooperation in complex networks precludes the existence of a specific behavioral mechanism and a particular network interaction structure [30]. This interaction structure often exhibits multidimensional features such as relationships of different types or temporal dynamics.
Against this background, we studied the cooperation dynamics under a behavioral mechanism based on generalized reciprocity, in a network consisting of multiple dimensions, each modeled by a random graph. The model, which is a generalization of the one introduced in [48], provides new insights on the role of the network structure on the promotion of cooperation in complex multidimen- sional networks. In particular, we demonstrated that a multidimensional structure may support cooperation within the individual network dimensions, even when the benefit-to-cost ratio in the considered dimensions is below the threshold required for cooperation (when observed in isolation). This observation may explain the existence of cooperation in systems where the network parameters itself would promote defection -a latent structure (i.e. other dimensions) may exist that acts as a support to the observed dynamics.
We also discussed the connection between the studied behavioral mechanism in the multidimensional network and reinforcement learning, by interpreting the act of presence of agents in the dimensions of the multiplex network as a selection of a strategy (from a predefined set of strategies), where the strategy selection is applied in relation to agents' internal state. In this context, we introduced a simple and intuitive rule for modeling the agents' interactions in the different dimensions (i.e. their presence across dimensions). The experiments were performed both on an multidimensional extension of the random graph models, and on a real-life dataset. As a general observation, the cooperative contributions of the individual agents concentrate in the dimension which is most favorable for the existence of cooperation.
An interesting direction for future work is the study of more general behavioral mechanisms in the spirit of the "exploration vs. exploitation" discussion in reinforcement learning. In this context, it will be valuable to study not only the steady-state, but also the transient behavior in the network. As a final note, the model can also be used as a starting point in the examination of network formation based on generalized reciprocity, where the neighborhood of each agent in each dimension can be seen as the possible final outcome of a rewiring process that is determined by the dimension update rule.