{'fraction_non_alphanumeric': 0.03506407648494711, 'fraction_numerical': 0.013857811228641171, 'mean_word_length': 4.573837868480726, 'pattern_counts': {'":': 0, '<': 2, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Evolutionary game theory and the evolution of neuron populations, firing rates, and decision making

Recently, there have been calls to apply evolutionary game theory to neuroscience. Ours, to the best of our knowledge, is the first application of dynamical evolutionary games to decision making in neuroscience. Firing neurons are the players in the game. The strategy of the players is their firing rate. Neurons with equal firing rates define a population. The neurons (players) do not know the rules of the game, they do not know what the reward is, they are not required to be rational and they do not even know they are playing the game. The interactions among the neurons is inhibitory. The theory confirms experimental data about decision making in vision (to move the eyes or not to a known target) in that: (i) A parameter of the game model (variance in the size of the neuron populations that participate in the game) determines how many distinct populations of firing neurons participate in the decision to move the eye to a target; (ii) if the populations of firing neurons are localized, then the solution of the game dictates how many loci in the brain participate in the decision to move the eye; (iii) the theory clarifies the difference between ultimate (evolutionary) factors and their effect on proximate (immediate) factors that influence decision making in the brain; (iv) as expected, the theory predicts that quick decisions are associated with more errors and slow decision are associated with fewer errors—however, the theory ties these factors to the evolution of brain decision making processes.


Introduction
Recently, game theory has been applied to decision making in neuroscience. One of the fundamental questions (that is much overlooked; see Glimcher, 2002) is the relationship between ultimate (evolutionary) and proximate (immediate) causation of behavior. Since the 1970s, evolutionary game theory has advanced the view that Darwinian evolution does not lead to fitness optimization (where optimization is defined as maximizing some objective function with respect to constraints). Instead, evolution yields solutions of games (where the solution is usually suboptimal). This lead to a profound change in the view of the outcome of Darwinian evolution-the latter, it was argued, does not necessarily maximize fitness. In applications of optimal control theory to ecology, there is a single organism on which natural selection acts to maximize its fitness. When there is more than one organism with its own objective function, then we have a game (see Brown and Vincent, 1987;Cohen, 1987). Here, we introduce a specific approach to decision making theory in neuroscience: dynamical evolutionary games. Decision making is mechanistic and in the context of a game, in which neurons are players. We also make clear distinctions between proximate and ultimate factors that influence neuronal responses to stimuli and eventually to decision making.
Evolutionary game theory involves players, rules, strategies and rewards. Each player attempts to maximize its reward by adopting a strategy value, according to the rules, and in response to other players' strategy values. In evolution by natural selection, players are individuals (proteins, cells, whole organisms). The rules are inheritance, phenotypic variation with respect to (inherited) adaptive traits, and natural selection. The strategies are values of phenotypic traits, such as anatomical features and acuity of perception. The rewards are fitness. The latter defines the survival of phenotypic traits through changes in their frequency and density in the population of firing neurons.
With respect to neuroscience, we make the following definitions. The players are single neurons. Their strategy is their firing rate. The reward is fitness; namely, the firing rate that results in maximizing survival of the strategy in the face of interactions with all other neurons with similar maximizing processes. Possible adaptive mechanisms that lead to maximizing fitness are acuity of perception and speed and accuracy of response to stimuli. The solution to a game depends on what one wishes to define as a solution. Thus, in games one talks about a solution concept, which from hereon we call solution. Solutions require that the outcome of the game obeys some definition of stability. One such concept is the Nash equilibrium (Nash, 1954), in which none of the players can gain by changing their strategy unilaterally.
Our goal is to apply a specific method borrowed from evolutionary game theory (Vincent and Brown, 2005) to answer the following questions: (i ) What is a reasonable choice of a strategy? (ii ) What defines interactions among neurons? (iii ) What is an appropriate definition of a solution? (iv ) How many distinct populations of neurons does the solution dictate? (v ) What is the size of these populations? (vi ) How may the solution of the game relate to decisions to respond to a stimulus?
We define a population as a collection of neurons that displays a distinct strategy value (firing rate). If strategies (different firing rates) are localized, then (iv ) answers the question of how many distinct population centers in the brain emerge from the solution of the game. There are fundamental differences between economic and behavioral games on the one hand and evolutionary games on the other. In evolutionary games, the players do not know the rules, they do not know what the rewards are, they are not required to be rational and they do not even know that they are playing a game. As in some other games, evolutionary games do not allow cooperation (e.g., negotiation). Consequently, evolutionary games are mechanistic-no self-awareness is involved.
Maynard- Smith and Price (1973) and later Maynard-Smith (1982) proposed the concept of Evolutionary Stable Strategies (ESS) as a solution to evolutionary games. Because evolutionary games obey Darwinian rules (Darwin, 1859), the outcome of evolution by natural selection is not necessarily the best adapted organisms. It is the group (community) of phenotypes with a set of trait values such that other phenotypes with different values (mutants) cannot invade the community and survive. In the game-theory vernacular, this community is called the coalition (not to be confused with the outcome of negotiations). The stable state of strategy values and population densities in which mutants cannot invade and coexist with the coalition is called ESS.
Two questions then emerge: what is the set of strategy values that make up the ESS and how many different strategy values coexist at ESS? Since 1973, a whole field of evolutionary games has emerged (Fudenburg and Levine, 1998;Hofbauer and Sigmund, 1998;Samuelson, 2002;Nowak and Sigmund, 2004).
Here, we introduce the so-called smooth dynamical evolutionary games (as opposed to matrix games). In particular, we follow the approach developed by Vincent and Brown (2005). To the best of our knowledge, ours is the first attempt to apply this specific game theoretic approach to decision making through interacting neuron populations. Consequently, our approach is mostly theoretical. Yet, we will address some qualitative predictions of the theory and cite published results that support or refute these predictions. We assume that most readers are not familiar with evolutionary game theory. Therefore, our exposition of the subject is at a tutorial level. Refer to Vincent and Brown (2005) for a complete treatment of the subject. Recent applications of evolutionary games include cancer research (Gatenby and Vincent, 2003) and Darwin finches (Vincent and Vincent, 2009).
The manuscript is organized thus: In section 2 we introduce the concepts upon which the remaining of the manuscript depends. We then define ESS and temporary ESS which we dub TESS (section 2.1), outline how one finds ESS (section 2.2) and discuss the important subject of strategy dynamicshow strategies change over time (section 2.3). Finally, we discuss the meaning of winners and losers in the context of evolutionary games (section 2.4). In section 3, we show how the game, with its dependence on parameter values, determines how many distinct populations of firing neurons survive the game. In particular, in section 3.1 we introduce a mathematical model that we rely upon. The model is used to fit (qualitatively) experimental results of one and two strategies to the model (section 3.3). We close with discussion (section 4), where we outline the potential contribution and limitations of the theory to neuroscience and potential future work.

Evolutionary game theory in the context of firing neuron populations
Here, we define and explain the concept of ESS and a related concept which we call Temporary Evolutionary Stable Strategies (TESS). We then outline a method to calculate them. In the following sections we will show how this concept may be used to describe a decision making process based on identifiable populations of firing neurons. Dynamical evolutionary games as formulated here were introduced by Vincent et al. (1993) and are explained in more detail in Vincent and Brown (2005). Some rigorous aspects of the theory may be found in Vincent et al. (1996) and Cohen et al. (2000). Games with multiple strategies are discussed in Brown et al. (2007). An alternative relevant approach to evolutionary games was introduced by Cohen (2003aCohen ( ,b, 2005Cohen ( , 2009a Here we introduce some fundamental definitions: Strategy value (u i (t)) A value on the real line, at time t, that controls the magnitude of some response of collection i of neurons to a stimulus.
We shall use u i as the firing rate of a neuron. This firing rate affects the neuron's and other neurons' firing rates through excitation and inhibition. The magnitude of the excitation and inhibition is mediated through some function of u i . In this manuscript we discuss inhibition only.
Virtual Strategy (v) The set (function) of all possible strategy values.
Population (x i (t)) The number of collection i of firing neurons that share the same strategy value (firing rate) at time t.
These definitions imply that if we have n populations of neurons participating in a single decision, then we have Populations ( x(t)) Defined as Corresponding to these populations, there are n strategy values (on a continuum of a single phenotypic trait which we call strategy); thus we have For each x i (t) there corresponds a strategy u i (t). To simplify the notation, and when unambiguous, we will write u i instead of u i (t) and so on for the other definitions. If u i are localized, then we recognize distinct population centers of neurons of the brain. Note that a population is defined by its function, as opposed to by the location of its member neurons. A functional population is then defined by the value members of the population (neurons) share with respect to strategy values (firing rates). A strategy value determines the function of neurons.
To implement dynamical games through differential equations, we make the following assumptions: (i ) The populations are large enough so that the addition or subtraction of a single neuron to a population may be considered relatively small. (ii ) If the strategies are localized, the response of an individual neuron is independent of its location within the local population. (iii ) The statistical moments of the response of the population of firing neurons (e.g., mean, variance) characterize the population trajectory. Assumption (i ) implies that changes in population sizes can be approximated by smooth functions. Assumption (ii ) implies that we can ignore the location of individual neurons in a specific area in the brain. Assumption (iii ) implies that we may ignore stochastic effects. Essentially, these assumptions hold when populations are large and the locations they occupy are small. The functions we use must have at least second derivatives.

Definition of ESS
Throughout, we discuss populations of neurons. The ideas can be applied to single firing neurons. In such a case, the firing rate of a single neuron is its strategy and it is equivalent to a population of neurons.
To simplify the discussion, we assume a single strategy and any number of populations, n. Multiple strategies are discussed in Brown et al. (2007). Again, a population is defined by a distinct value of a strategy. Membership in a population is not necessarily fixed. For each response to a stimulus, different neurons may be members of different populations. However, during a single response to a single stimulus, membership remains unchanged. If the firing rate of a neuron is genetically determined (i.e., the neuron will always fire at the same rate), then membership in a population is permanent. Some neurons might belong to a population permanently and others temporarily.
Let x i (t) be the number of neurons firing with rate u i (t) at time t. We write the rate of change of x i generically thus: where primes denote derivatives with respect to time. In evolutionary ecology, H i is called instantaneous fitness function. When H i > 0 the population grows, when H i < 0 it declines, and when H i = 0 it is at equilibrium, which may or may not be stable.
In what follows, we assume that there exists an equilibrium solution (i.e., If some of thex i > 0, we say that the system is in neuronal equilibrium. Otherwise, it is not. If small perturbations to¯ x result in the system returning arbitrarily close to this equilibrium, then we say that the neuronal equilibrium is (asymptotically) stable. For a system in stable neuronal equilibrium, we can orderx i from largest to smallest so that some of thex i (or none at all) will equal zero. Suppose that 0 < r ≤ n elements of¯ x are positive and n − r are zero. This allows us to define ESS thus:

ESS
The vector¯ x r is an ESS for the stable neuronal equilibrium¯ x if for any m > r and any u,¯ x is a stable neuronal equilibrium.
Consequently, any population with firing rate u that is different from those we find for ESS cannot coexist when the system is at ESS. The definition also implies that we have to establish the right number of positive populations, r, that participate in ESS. The ESS solution determines the values of u for each population, how many distinct populations must participate in the process and the size of each population. If these populations are localized, then r is the number of centers in the brain that must participate in the game for the coalition to be an ESS. The question is how we establish the values of¯ x and u that guarantee that we found the ESS. We address this question in the next section. It turns out that in addition to ESS, we will be interested in how the system approaches arbitrarily close to ESS from some initial conditions. We call this period of approach the Temporary ESS (TESS). The TESS will provide a clue as to the rules for decision making based on x and u.

Finding ESS
To provide a recipe for finding ESS (and TESS), we need to first define the so-called G-function. Recall that u defines a set of n points on the strategy curve. Each of these points corresponds to a particular x. We define for all t, where H i is defined in equation (1) (see Vincent et al., 1996).
Note that u i are specific strategy values. However, as we shall see, u i change in time. The regulating mechanism (inhibition) drives all values of u to points where each value of x is at its maximum with respect to all of uthis in essence is what makes the formulation a game. Consequently, we need to introduce a virtual variable, v, that will assist in determining these maximums. Thus we write the G-function as G (v, u, x). Only when v = u i do we achieve the equality in equation (2). A further motivation for using v is provided below. We present the following theorem without a proof (see Vincent et al., 1996).
To untangle the notation, note that v is a curve of the strategy values. The shape of the curve changes as u "seeks" its maximum. u are points on the curve v that correspond to the populations x. As we shall see, v plays a role in searching for the maximum of G.

Strategy dynamics
For the ith population of neurons, u i represents the population average value of the phenotypes u ij . With some further assumptions-that are irrelevant here (but see Vincent et al., 1993)-we obtain where σ controls the speed with which x and u get to the ESS, (¯ x,¯ u), from the initial values for the system defined in equations (1) and (3). The notation in equation (3) implies that one first takes the derivative of G with respect to v and then inserts the values of u i .
Upon examination of equation (3), it becomes clear why we treat v as a virtual variable. It serves as a place holder for the derivative of G after which we replace its value with the current value of u i . Note that equation (3) represents a gradient maximum seeking algorithm: at every instant of time, u moves up the steepest slope that is determined by G(v, u, x). Equations (1) and (3) with appropriate initial conditions define the dynamical smooth evolutionary game. It is dynamical because the solution is obtained over time; it is evolutionary because it refers to players (neurons) that exert control on all other players (including themselves) to obtain fitness values. It is a game because the control mechanism (u) drives each x i to its maximums possible size simultaneously. Thus, the maximum that x i (t) can reach must simultaneously depend on those maximums that all x j (t) reach with respect to all u i . These maximums meet the definition of ESS. The road to ESS is TESS.

Winners and losers
In the search for ESS, there are winners and losers. Winners are those populations that remain positive at ESS. These are also the populations that participate in TESS. Populations that may begin or enter the process and then become zero constitute the losers. In this sense, in evolutionary time, if two populations end up participating in the decision to move the eyes to a target in the field of vision, and both populations participate in achieving ESS, then both are winners. These will be the two populations that participate in the TESS game.

Decision making with TESS
We shall use the above theory to make some qualitative predictions about neuronal responses during decision making and examine the extent to which these predictions agree with experimental results.
The search for ESS, and therefore TESS, proceeds as follows: We first construct a mathematical model of the system. Some properties of the system (as discussed below) determine how many populations with distinct firing rates participate in the ESS. Next, we try a solution with a single population. If we find a stable equilibrium that meets the conditions specified in the maximum principle above (i.e., u that maximizes G at G = 0), then we have found ESS. The trajectories of x and u from some initial conditions to ESS define TESS. If we do not find ESS, we try a solution with two populations, and then three, and so on.

One-population ESS
Here we show the evolution of firing rates, and therefore decision making, from two populations to one.

A mathematical model of the game
We modify the Lotka-Volterra competition model to construct the mathematical model of system (1) and (3) (see Vincent and Brown, 2005). Suppose that at t = 0, x 1 (0) and x 2 (0) start to coevolve according to the dynamics and We will show that if coevolution dictates one population at ESS, then both populations merge, in evolutionary time, into a single population. By definition, a single population is comprised of those neurons that share the same firing rate at ESS. Therefore, only one population at TESS is involved with the neuronal decision mechanism. Consequently, a response may be triggered by a threshold process (Hanes and Schall, 1996) or by achieving a plateau (where H = 0).
The function k(u i ) determines the population size at t, as a function of the strategy value (firing rate), u i . When k is constant, it is usually called the carrying capacity. The explicit form of k is I was first proposed by Roughgarden (1983). Here k m is a scalar; it is the population size of x i when u i = 0. To avoid extra parameters in the system, we adjust the firing rate such that k(u i ) is maximized when u i = 0. Often a simple translation-moving u i to u i -u * where u * is the value at which k(u i ) is maximized-will do the job.
The value of σ k determines the plasticity of deviation from the value of u (= 0) that maximizes k(u i ). When σ k is small, a small deviation from u i = 0 results in a large decrease of x i at t. Figure 1 illustrates the sensitivity of k(u i ) to changes in k m and σ k .
The role of σ k in contributing to the potential maximum of x i permits interesting predictions of the present theory: (i ) In situations where quick decisions, as opposed to accurate ones, are crucial to the fitness of the organism, σ k will be large because small changes in u i cause small drop in k(u i ). (ii ) When quick decisions are not crucial (so that, for example, accuracy plays an important role), then σ k will be small. (iii ) These conclusions are predicated upon the assumption that small (or large) σ k is associated with some neurophysiological cost-otherwise, all σ k will be rigid for decisions that relate stimuli that require different responses (fast and inaccurate, slow and accurate). If x i are spatially concentrated, then the theory predicts that large k m should be associated with large local concentration of populations.
If the parameters in equation (6) can be measured (an open question at this time), then the theory predicts that, for example, the neurophysiological response of prey to the detection of predators (flee) should be associated with large σ k -you better run away quickly even if you mistakingly thought that a predator was present. The response to say (usually ample) food detection should be associated with small σ k -you can take your time making a decision, but you better not eat poisonous food.
Now α(u i , u j ) in equations (4) and (5) is a function that translates the values of u i and u j (the firing rate of populations i and j) to the magnitude of inhibition that is exerted by x i on x j . For example, α(u 1 , u 1 ) describes self-inhibition. Note that α depends on both strategy values simultaneously. The explicit form of α is It was proposed by Brown and Vincent (1987). To interpret the roles of the parameters, note that the middle term in the right hand side of equation (7) is at its maximum when u i = u j . This means that like inhibits like the most. This is akin to the interpretation of competition in ecological communitiesindividuals of the same species compete with each other more than with individuals of another species. Because we do not expect the inhibitory effect to be symmetric away from its maximum, we add the parameter b, which scales the asymmetry. The last term in equation (7) ensures that the maximum of α(u i , u j ) is 1. Finally, the interpretation of σ α is equivalent to that of σ k . Figure 2 illustrates the interactions among these parameters. To write the strategy dynamics, define Note that upon replacing v with u i , the right hand side of equations (4) and (5) are in fact G (u i , u, x) , i = 1, 2. Now we write the strategy dynamics as and A solution of the system (4) to (9) with appropriate initial conditions solves the game. In general, n can be larger than 2. For example, for n = 3 we have six differential equations to solve, three for x and three for u.
The numerical solution ( Figure 3) indicates that x 1 at the presumable ESS is larger than x 2 . Consequently, a measurement of the total firing rate from x 1 will be higher than that from x 2 . Although x 1 and x 2 start from identical initial conditions-which represent the background noise-their trajectories separate because u 1 (0) = u 2 (0). If the neuronal steady state that both x and u reach is ESS, then by our definition we have a single population because their firing rates u i are equal. One can produce different trajectories of x 1 and x 2 , but one will always get a single u, albeit with different roads to ESS, depending on the initial conditions. Recall that the initial conditions represent the background noise just before stimulus presentation at t = 0.
Our next task is to verify that the equilibrium shown in Figure 3  with respect to u 1 and verifying (with second order derivatives) that the solution is in fact a maximum G = 0 when v =ū 1 and x =¯ x.
An alternative ad hoc method is to numerically solve the system (1) and (3) for a very large t to obtain¯ u = [ū 1 ,ū 2 ] and¯ x = [x 1 ,x 2 ] as t → ∞. If we have a single strategy at ESS, thenū 1 =ū 2 . Next, substitute these values into G (v, u, x) and plot the resulting function. If its maximum is achieved at (¯ u,¯ x) and the value of this maximum is zero, then we have found ESS. To illustrate this, we find from the long-time numerical solution of the system defined by equations (4) to (9) with the parameter value in (10)  and plot the results (Figure 4), verifying that the solution is a single population at ESS.
Suppose that we can obtain numerical estimates of the parameters of system (4) to (9). We then solve for ESS. If the solution dictates a single population of firing neurons, then the theory predicts that the neuronal decision making must be based on either a threshold or a plateau criterion. It cannot be based on, for example, differences in firing rates from two populations of neurons. Note that the road to ESS (what we call TESS), between (scaled) firing rate values of 0 and 3 depends on the initial conditions. Now if we assume that the variation of the background firing rate is small, then we have a good reason to always start TESS from identical initial conditions.

Two-population ESS
The results here illustrate the importance of σ k in determining the number of participating populations in the ESS solution, the differences in firing rates between the two coexisting populations and the shape of TESS.
Suppose that we keep all parameter values as in (10), except for σ k = 8 (to speed the approach to ESS, we also change σ = 4). The larger value of σ k implies that the decision needs to be fast, at the expense of errors; the smaller value of σ k implies that the decision must be accurate, at the expense of speed. In such a case, we get coexistence of two populations at ESS ( Figure 5). We draw the reader's attention to the qualitative similarity The right hand panel shows the firing rate (compare to Figure  3).
between u (right panel of Figure 5) and the firing rates of neurons in the frontal eye field (Schall and Hanes, 1993;Sato et al., 2001;Cohen et al., 2009a), lateral intraparietal area (Thomas and Paré, 2007), and superior colliculus (McPeek and Keller, 2002) during visual search decisions when the target of search is located inside the receptive field of the neuron (larger firing rate) versus outside the receptive field (smaller firing rate). If the two populations are located in different areas of the brain, then we get two distinct centers in the brain that participate in the decision. Here, the decision may be based on the difference between the firing rates u 1 and u 2 . This decision principle has begun to be explored (Mazurek et al., 2003;Purcell et al., 2008). One should not get too attached to the shape of the TESS; many different shapes, that may fit experimental results, can be produced based on fitting parameter values. The fundamental result is that we end up with two distinguishable populations; and as Figure 5 illustrates, even if the background firing rates (initial conditions) are close, the u will diverge. Again, if the strategies are localized, then we identify two centers in the brain that participate in a single decision.
For posterity, we show the ESS in Figure 6.  Figure  4).

Application to experimental results: strategies and decisions
To illustrate the applicability of the theory, we fit TESS to data from Cohen et al. (2009a). In this study, monkeys were trained to search for a target stimulus among an array of distractor stimuli, and to indicate the location of the target by moving their eyes to it. Single neurons in the frontal eye field showed higher firing rate when the target was inside their receptive field versus when a distractor stimulus was inside their receptive field. Thus, these neurons showed activity correlating with the decision about the location of the target. Figure 7 shows a qualitative fit to the data in ( x 1 (0) = x 2 (0) = 30, u 1 (0) = 6.5, u 2 (0) = 5.5 Apparently, two strategies, and therefore two populations of firing neurons, are involved in visual search decision making. Cohen et al. (2009a) showed that the decision about the location of a search target correlates with the difference in the firing rates among these two populations.
A different population of neurons in the frontal eye field increases their firing rate leading up to saccadic eye movements to a target in their movement field, but fire at baseline during saccades outside their movement field. Figure 8 shows a qualitative fit to one of these movement neurons using a the scaled firing rate of a frontal eye field movement neuron when a monkey made a saccade into its movement field in a visual search array ). The lower curve shows the firing rate of that neuron when the monkey made a saccade to a location opposite the movement field; the neuron fired at its baseline rate under this condition. Therefore, we have one strategy only. The smooth curve is the fit (by eye) with the values shown in (12).
single-population ESS with the parameter values r = 0.2, k m = 100, σ k = 3, b = 4, σ α = 4, s = 1, (12) x 1 (0) = 30, u 1 (0) = 6.5, u 2 (0) = 0.5 The upshot is that frontal eye field neurons involved in discriminating the search target from distractors may use two strategies, while those neurons involved in preparing the eye movement may use one. These two types of neurons-visual neurons, as shown in Figure 7, and movement neurons, as shown in Figure 8-have been described as forming two stages of processing during visual search decisions: target selection and movement preparation (Schall, 2004). Note that a decision with one strategy requires exceedance of a threshold (Hanes and Schall, 1996). Decision making with two strategies may require a threshold of separation of firing rates between the two populations (Purcell et al., 2008).
In experiments that involve visual search decisions, the difficulty of the task can be manipulated by varying the number of distractors presented with the target or the similarity between target and distractors (Treisman and Gelade, 1980;Duncan and Humphreys, 1989;Sato et al., 2001;Wolfe, 2007;Cohen et al., 2009a). Experiments using more difficult search tasks confirm that the decision to move the eye to the target takes longer (σ k is smaller). The theory predicts that the easier the visual search task (e.g., few distractors or low target-distractor similarity), the larger σ k is and the sooner the strategy trajectories of TESS bifurcate (which may lead to the decision to move the eyes to the search target). When the search task is difficult (e.g., many distractors or high target-distractor similarity), σ k is smaller. Figure 9 illustrates this principle and, qualitatively, experiments 0 150 300 0. confirm the theoretical predictions. This prediction is not limited to visual search decisions; indeed, the ubiquitous speed-accuracy trade off is not limited to mammals (e.g., Skorupski et al., 2006) or to visual decision making (e.g., Rinberg et al., 2006). The ultimate agreement between experiments and theory can be achieved only when (and if) σ k can be measured (note that σ 2 k is the variance of the population of decision-related neurons due to changes in strategy values).

Discussion
A rich literature on the theory of the neural basis of decision making has emerged recently (Reddi and Carpenter, 2000;Usher and McClelland, 2001;Wang, 2002;Mazurek et al., 2003;Ratcliff and Smith, 2004;Smith and Ratcliff, 2004;Brown and Heathcote, 2005;Bogacz et al., 2006;Wong and Wang, 2006;Beck et al., 2008;Doya, 2008;Ganguli et al., 2008;Smith and Ratcliff, 2009). This theory has been grounded in the principle that sensory evidence is accumulated in brain areas involved in decision making. Here, we introduce a new approach to decision making theory in neuroscience: dynamical evolutionary games, in which neurons are players.
Applications of game theory in neuroscience have dealt with decision making of individuals (Barraclough et al., 2004;Sanfey, 2007;Lee, 2008), particularly in social neuroscience. Recently, three reviews have called for applications of game theory from ecology (Glimcher, 2002;Glimcher and Rustichini, 2004;Schultz, 2004) to understand the neural mechanisms of decision making. In this paper, we applied principles from dynamical evolutionary games to well-studied decisions in the neurophysiology literature.
Neurophysiological studies have identified areas of the brain involved in visual decisions, particularly in prefrontal cortex (Schall and Hanes, 1993;Kim and Shadlen, 1999;Sato et al., 2001;Cohen et al., 2009a), parietal cortex (Roitman and Shadlen, 2002;Thomas and Paré, 2007;Churchland et al., 2008) and superior colliculus (Horwitz and Newsome, 1999;McPeek and Keller, 2002). In these studies, single neuron firing was recorded and correlated with decisions about a visual stimulus. In our dynamical evolutionary game framework, we associate the firing rates of these neurons with the population strategy, u. This population strategy corresponds to the firing rate of a single neuron upon repeated presentation of a visual stimulus in these experiments. The theory is not restricted to visual decisions, but we compare theory to experiment in this domain because it is one of the best-understood and represents our research interest.

Interpreting parameters
As we have described above, u corresponds to the firing rate of individual neurons in a population and x corresponds to the number of firing neurons in that population. Here, we speculate about the relationship between other components of the model and neurophysiological processes; clearly, experiments must provide evidence for mappings between parameters in dynamical evolutionary games and neurophysiological processes.
We showed that α(u i , u j ) translates the values of u i and u j to the magnitude of inhibition exerted by x i on x j . Thus, α can be associated with several factors that control inhibition, including neuromodulatory systems (Schultz et al., 1997;Hasselmo and McGaughy, 2004;Aston-Jones and Cohen, 2005), and with mechanisms of inhibition, including shunting inhibition (Mitchell and Silver, 2003). The effects of inhibition have been observed in studies that measured responses from multiple neurons simultaneously during decisions (Constantinidis et al., 2002). Inhibition has also been proposed as a mechanism for synchronizing neural activity (Van Vreeswijk et al., 1994;Bush and Sejnowski, 1996), which may be important for decision making.
As described above, the parameter σ k governs the speed and accuracy of the decision. When σ k is large, response time will be fast but there will be errors. When σ k is small, response time will be slow but decisions will be accurate. We completely ignored the relation between σ k and σ α and how their interactions may affect the number of populations that participate in ESS. We also ignored interactions among other parameters. Yet, the direct effect of σ k on the size of the populations of firing neurons seems to be a good starting point.

Limitations
There are several assumptions that we make using dynamical evolutionary games. Chief among them, is that the size of the populations of firing neurons are large enough to justify a deterministic approach with smooth functions. The theory as presented here is not appropriate for small populations, where stochastic effects play a crucial role in the dynamics of the system (see Cohen, 2009b).
As formulated, the theory emphasizes inhibition. A natural extension would be to use predator-prey models with competition. In such systems, some populations of firing neurons may be inhibitory (predators on prey) and some excitatory (prey on predators). Other useful relationships that can be borrowed from evolutionary game theory include host-parasite.
The concept of ESS does not imply that evolutionarily, populations of firing neurons achieve it. It is a state toward which the system is moving.
An important extension of the theory would be with multiple strategies (Brown et al., 2007). Such an extension does not require a major shift from the theory discussed here. It does imply some complications in notation and computations.
Aside from the inability of smooth evolutionary game theory to deal with small populations, there are additional limitations (or perhaps difficulties). First, one has to come up with appropriate models-often a difficult task. Second, one has to establish a link between the parameters of the mathematical models and experimental measurements. Third, one has to come up with an appropriate G-function; currently, there is no formal approach to derive it.

Predictions
How well does the theory predict experimental results? We showed above a qualitative fit of the dynamical evolutionary game strategy to firing rates of frontal eye field neurons participating in visual search decisions. While not the goal of this paper, quantitative fits are a matter of fitting parameters. Now, consider equations (8) and (9), the strategy values. These two equations differ only in their initial conditions. Thus, a prediction of the model is that firing rates of neurons participating in decisions will be governed only by the parameters in the model. Future experiments should determine which parameters are relevant for generating neuronal activity such as that in Figure 7.
The evolutionary game makes predictions about the response time of a subject during easier versus harder decisions. A harder task may be associated with closer initial conditions between u 1 and u 2 , and therefore result in later separation between those firing rates. This has been observed experimentally (Sato et al., 2001;Cohen et al., 2009a). A novel prediction of the theory is that when the decision is harder, the population of neurons making the decision is smaller (governed by k(u)). What controls the membership of a neuron in a population is the solution of the game. We show that stronger inhibition may be required for harder decision making.
Albeit not discussed in this manuscript, an intriguing possibility is that cortical neurons do not fire randomly, as has been reported (e.g., Shadlen and Newsome, 1998), but chaotically (Hansel and Sompolinsky, 1996). This means that the firing rate (strategy) of populations of neurons is deterministic and different results are obtained from different initial conditions. Extensions of evolutionary game theory to chaotically firing neurons exist (Vincent and Brown, 2005), but much work remains to be done.