{'fraction_non_alphanumeric': 0.028414108358007378, 'fraction_numerical': 0.008830710092982183, 'mean_word_length': 4.781381381381381, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Effect of Synchronous Incoming Spikes on Activity Pattern in A Network of Spiking Neurons

Although recent neurophysiological experiments suggest that synchronous neural activity is involved in some perceptual and cognitive processes, the functional role of such coherent neuronal behavior is not well understood. As a first step in clarifying this role, we investigate how the temporal coherence of certain neuronal activity affects the activity pattern in a neural network. Using a simple network of leaky integrate-and-fire neurons, we study the effects of synchronized incoming spikes on the functioning of two mechanisms typically used in model neural systems, winner-take-all competition and associative memory. We demonstrate that a pair of switches undergone by the incoming spikes, from asynchronous to synchronous and then back to asynchronous, triggers a transition of the network from one state to another state. In the case of associative memory, for example, this switching controls the timing of the next recalling, whereas the firing rate pattern in the asynchronous state prepares the network for the next retrieval pattern.


I. INTRODUCTION
Synchronous neural activity, that is, coherent spiking behavior, is widely observed in various cortical areas, and is often stimulus-specific and task-specific [1,2]. These observations suggest that synchronous neural activity may play a significant functional role in cognitive and perceptual processes [3,4]. One proposed possibility is that such a coherent spiking is the signal for the binding of different attributes of an object into a single percept. To this hypothesis, we have to first clarify the neural mechanism underlying this coherent activity.
Over the last decade, a considerable number of studies have been focused on determining the dynamical mechanisms underlying synchronous firing in neural systems [5,6,7,8,9,10,11] , for example, elucidating the neural mechanisms that enable cortical neurons to synchronize and how the neural network controls the degree of synchrony in a stimulus-dependent manner. [12,13] The next important issue is to clarify how such coherent activity contributes to information processing. However, there have been relatively few studies addressing this issue. [14,15] Considering the above situation, in this paper, we explore the possible functional role of synchronized neural activity.
Many electrophysiological experiments have shown that cortical neurons do not always synchronize and that the degree of synchrony is dynamically modulated by a number of factors, such as external stimuli and the internal state. In fact, recent experiments show that the degree of coherence among spikes is modulated by attentional shifts. [1,16] Typically, it is observed that such coherent activity transiently lasts for a few hundred msec. These facts indicate that the switching between asynchrony and synchrony provides a crucial signal for information processes in the brain. In developing a theory of information processing in the brain, it is necessary to understand how modulation of the level of synchrony influences the function of the neural network. In particular, we study the influence of synchrony-asynchrony switching in incoming spikes on the activity pattern on a neural network, resulting from two mechanisms typically employed in model neural networks, winner-take-all competition and associative memory.

II. WINNER-TAKE-ALL COMPETITION MODEL
Let us begin our analysis by considering winner-take-all(WTA) competition, which is commonly used as a computational model to determine the best choice among different possible outputs of the network. Is is believed that this competition mechanism provides a fundamental mechanism used in the brain. For example, various models using this mechanism have successfully reproduced columnar structures found in the visual cortex [17,18,19,20].
Through WTA competition, the only neuron receiving the largest input in a network becomes active as the 'winner'. Recent simulations have revealed that such competition is extremely sensitive to the coherence among neuronal action potentials [21]. We study this phenomenon theoretically and explore the possible role of synchronization in the functioning of neural networks.
A schematic of our WTA model is displayed in Figure 1(a). As the single neuron model, we adopt the leaky integrate-and-fire neuron model, in which the basic equation determining the membrane potential V i of the i-th neuron is given by where R m = 100MΩ and τ m = R m C m = 20ms are the membrane input resistance and the membrane time constant, respectively. Whenever V i (t) reaches the threshold voltage V th = −50mV , a spike is generated, and V i (t) is instantaneously reset to the resting potential, V rest = −65mV . The total current I i (t) consists of the sum of all inputs to the neuron, given by where N is the total number of neurons, t i is the time at which the i-th neuron generates a spike, and ∆ is the synaptic transmission delay. The postsynaptic potential is expressed simply as the alpha function J(t), which is given by where τ is the time constant of a synapse, and Θ(t) is the step function, given by The current I inh i (t) represents the effect of all-to-all uniform inhibitory couplings, which is responsible for the WTA behavior of the network. We also assume that each neuron is self-connected by an excitatory synapse, I selfex i (t). This single neuron in our model can be regarded as representing a population of the real neurons comprising a single columnar structure. Therefore, it is quite natural that such excitatory self-connections are introduced, corresponding to the mutual excitatory synaptic connections existing within an actual column of this kind.
We assume that each neuron in the network receives input from two types of external sources. One is the synchronous/asynchronous (Syn-Asyn) input layer, in which the level of coherence among the input spikes can be controlled to examine the effect of the synchronyasynchrony switching of incoming spikes on the activity pattern in a neural network. This uniformly applied current is generated according to where t syn−asyn is the time at which neurons receive a spike from a neuron in the Syn-Asyn input layer and N input is the total number of neurons in this input layer. We consider the situation in which this input layer possesses two modes, an asynchronous mode and a synchronous mode. In the asynchronous mode, each spike is generated by a Poisson process. In the synchronous mode, 45 percent of the neurons in this layer fire periodically and synchronously with spike timings distributed in a Gaussian distribution of variance σ = 4ms. In both modes, we assume that the average firing rate is 25Hz.
The other external source is the pattern input layer, which is the neuron-specific current, given by Each neuron in this layer projects to one neuron in the output layer in a one-to-one correspondence. By controlling the activity of neurons in this layer, the biased current for each neuron in the output layer is determined, which affects the firing pattern selected through the WTA competition.
For these synaptic connections and external inputs, we use the parameter values ∆ = 1ms, 14nS, N input = 1000 and ∆I = 7.5pA. Figure 1(b) depicts a typical effect of synchronized incoming spikes on the properties of the WTA competition. Here the simulation was performed with a network of 100 neurons.
The system was started with the condition that the Syn-Asyn input layer is set in asynchronous mode, and in the pattern input layer, the currents I ext i (t) in the 1st through 40th neurons are active ( Pattern 'A'). Owing to the nature of the WTA competition, only those neurons that receive a larger input are active (as seen during the first 200 ms in Fig.1(b)).
After these currents in the pattern input layer turn off, the firing of the winners(Pattern 'A') is maintained. Next, even though another current pattern 'B' applying to some loser neurons(60-100th) is presented in the input layer, the firing pattern of winners remains stable (time steps 400ms -600ms). This result is typical behavior of WTA competition models.
With the situation as described above, let us consider a transient synchronized spike inputs with temporal switching modes in the Syn-Asyn input layer. In this case, as shown in Figure 1(b), the synchronized incoming spikes cause the old firing pattern to become unstable. Although the old firing pattern 'A' is maintained at first, the network eventually exhibits a collective synchronized oscillatory state. When the asynchronous mode in the Syn-Asyn input layer is recovered, a second firing pattern 'B' is realized through WTA competition, as determined by the activity of the pattern input layer. The other neurons(40-60th) that receive no pattern input do not fire even in a synchronous input period. It depends on the parameters whether or not these neurons fire without pattern input during synchronous input period. However, even if these neuron fire, the next pattern 'B' always is realized in the following asynchronous input period. Therefore, it should be noted that a transition of the firing pattern in the output layer is induced by synchrony-asynchrony switching in incoming spikes. This result means that the synchronized incoming spikes nullify the effect of the function of WTA competition, erasing the current pattern realized in the WTA competition, and thereby allowing it to make a transition to a new pattern prepared by the pattern input layer.

III. ANALYSIS
In order to understand the result of the numerical simulations described above, we now analyze the stability of the state selected through the WTA competition.
For the convenience of analysis, first we rescale equation (2). Using a transformation given by where parameters are rescaled as Next, assuming that the total number of neurons in these layers is sufficiently large and the variance of the synchronous spike timing σ = 0, the form of the input currents can be simplified as where T 0 is the period of the input spikes from Syn-Asyn input layer. ρ is the ratio of synchronized neurons.
From the results of simulations, we know that typically, winner neurons fire periodically and synchronously at the times s i = nT (n = 1, 2, ...) for the asynchronous mode in the Syn-Asyn input layer and s i = n(T 0 − θ) for the synchronous mode. Consequently, we consider only these solutions in this analysis and do not deal with other more complicated solutions.
To analyze these solutions, the unknown parameters T and θ should be determined in a selfconsistent way. In the asynchronous mode, integrating equation 4 between nT and (n + 1)T , we get a self-consistent equation for the period T , where N p is the number of the winner neurons. Solving this equation, T is determined and the spike trains of winner neurons are determined.
Similarly, in synchronous mode, we also derive a self-consistent equation for θ, and determine the spike trains of winner neurons.
Given the spike trains of the winner neurons, the time evolution of the loser neurons can be calculated. Let v n i (φ) denote the membrane potential of i-th neuron at the time s = T (n + φ), in which φ goes from 0 to 1. The return map from v n i to v n+1 i is given by The stability of the WTA state is ensured by the condition that none of the loser neurons fire, therefore, v ∞ i (φ) < 1 φ in [0, 1) for all loser neurons i .
As depicted in Figure 1 The parameter values used in Figure 1 is indicated by the +.
analysis is shown by the numerical simulation data in Fig.2. In addition, it also shows that the result is not too sensitive to both the synaptic delay∆ and the synaptic time constant τ .

IV. ASSOCIATIVE MEMORY
We next consider the case of the associative memory model [22,23], in which information is stored in the synaptic weight matrix, W ij . For the learning rule of the synaptic matrix, we adopt the Willshaw-type learning rule [24], defined by where P is the total number of the stored patterns, ζ where the factorN = j W ij i is the average connections of synaptic matrix W ij . Figure 3 depicts typical behavior of the associative memory model found in our numerical simulations, in which 10 patterns ('×' , '⋆' and 8 randomly generated patterns) are stored in the network of 1600 neurons. The 'key' pattern is presented as the activity in the pattern input layer, displayed as a 40x40-dot image. The 'retrieval' pattern is displayed as a 40x40dot image, in which the gray level of a dot indicates the averaged firing rate over 100 ms, normalized by dividing by the maximum firing rate. As shown in Figure 3, when a slightly distorted version of the pattern '×' , consisting of '×' plus some noise, is presented as a key pattern, the network correctly realizes the pattern '×' as the outcome of the associative memory functioning (0 -200ms). It is interesting that, if the spike trains in the Syn-Asyn input layer synchronize, the existing retrieval pattern '×' becomes unstable(600 -900ms), leading eventually to the synchronous oscillatory state as in the case of the WTA model.
Then, when the input spike trains become uncorrelated again, the network comes to exhibit the new retrieval pattern '⋆' in response to the key pattern presented by the pattern input layer(900 -1200ms). Hence, the synchrony-asynchrony switching of the incoming spikes plays a crucial role in triggering a change of the retrieval pattern, whereas the biased input from the pattern input layer prepares the network for the next retrieval pattern.

V. DISCUSSIONS
In the present paper, we have studied the manner in which the spike synchrony in the input layer can affect the activity pattern in a neural network. We find that the synchronyasynchrony switching undergone by the incoming spikes plays a key role in triggering a transition from one state to another state, even if the firing rate of incoming spikes is kept constant. For example, in the case of the associative memory model, the timing of the next retrieval can be controlled by such a synchrony-asynchrony switching of the external uniform input spikes, whereas the averaged rate pattern in the pattern input layer allows for the selection of the set of firing neurons for the next retrieval pattern. We will comment on a conventional way to reset activity patterns in WTA network.
Because of the well-known multi-stability in a WTA network, the transition in the output layer can also be achieved by changing the biased current from the pattern input layer. Therefore in Figure 1(b), if the input current from the pattern input layer for pattern B is sufficiently large, a switching from A to B occurs. This increase in the biased current corresponds to an increasing ∆I in Figure 2, which makes the WTA state unstable. By contrast with this conventional way, the important point of our model is that the firing rate of incoming spikes does not change during a transition in the output layer. The purpose of the present paper is to propose an alternative mechanism to control the neuronal activity in a neural network by altering incoming spike synchrony only.
Some multi-recording experiments suggest that spike synchronization preferentially occurs in relation to a purely cognitive event, where firing rate modulations were absent [1,25].
For example, it is shown that before the onset of voluntary movement, cortical neurons in the motor area transiently synchronize without changing the firing rate of each neuron. In addition, many studies in the recording of local field potential or electro-encephalographic activities also show that γ frequency oscillation is observed in association with upcoming movements. Therefore, some have suggested that these synchronous neural activities can be regarded as the representation of an anticipation or a preparation of an upcoming movement. In our understanding, such a synchronization in neural activity seems to play a role in triggering a transition to the next firing state for the upcoming movement. Hence, we believe that some of the observed synchronous activities are essentially relevant to our result in the present paper.
Next, we will comment on the fact that the same transition can be achieved by uniform rate modulation in the Syn-Asyn input layer. From a dynamical point of view, synchronyasynchrony switching constitutes a kind of perturbation leading to a transition from one attractor to another. It can be imagined that the same effect could be achieved through some other kinds of perturbations that do not involve synchrony among spikes, such as uniform modulation of the firing rate. To clarify this point, we investigated the effectiveness of such rate-modulated spikes numerically. In this case, instead of the synchronized incoming spikes, we used an abrupt change in the firing rate from 25Hz to 75Hz or 100Hz, but the timing of the change in firing rate is distributed according to a Gaussian distribution with variance σ (Figure 4a). Figure 4b displays the probability of a successful transition induced by such a modulation of the firing rate as a function of the width of the variance σ. We see that a change to 75Hz is insufficient to achieve a high probability of transition, even when the changes in firing rates are completely synchronous (i.e. σ=0). In the case that the firing rate is increased to 100Hz and σ is sufficiently small, the probability for a transition is comparable to the case of synchronized spikes (see Sync.25Hz in Fig.4b). Therefore, we conclude that a large and simultaneous increase in firing rate is required to realize a high probability for a transition. This implies that this modulation can practically be regarded as a kind of temporal coding.
Finally, we will point out that the synchrony-induced transition has different properties from the rate-modulation-induced one as mentioned above. As shown in Fig.4(c), the WTA state is still stable even with a high firing rate. This analysis is based on the assumption that the incoming spike rate is constant. Therefore, a sudden increase in the firing rate may cause a transition to another firing pattern. In fact, the numerical simulations show that a The dashed curves corresponds to the case of synchronized spikes. (c) Dependence of the stability of the state selected through the winner-take-all competition on the input spike rate in Syn-Asyn input layer and the pattern input current ∆I. Note that the constant rate for input is assumed.
large and almost simultaneous increase in firing rate over all neurons can trigger a transition in the output layer. On the other hand, by switching to a synchronous spike input, the WTA state becomes unstable. Therefore, the old firing pattern of the WTA state is eventually broken for the several synchronous spikes, whereas the old pattern is still maintained for the first synchronous spike ( Fig.1(b)). This result indicate that synchrony-induced transition mechanism is different from the rate-modulation-induced method. We believe that instead of rate-modulated type of perturbations, synchrony-asynchrony switching in incoming spikes may provide another sophisticated way to cause a transition of the firing pattern.