{'fraction_non_alphanumeric': 0.03850720949957591, 'fraction_numerical': 0.016573367260390163, 'mean_word_length': 4.648270575835968, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 4, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 5, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Homeostatic Plasticity Achieved by Incorporation of Random Fluctuations and Soft-Bounded Hebbian Plasticity in Excitatory Synapses

Homeostatic plasticity is considered to maintain activity in neuronal circuits within a functional range. In the absence of homeostatic plasticity neuronal activity is prone to be destabilized because Hebbian plasticity mechanisms induce positive feedback change. Several studies on homeostatic plasticity assumed the existence of a process for monitoring neuronal activity on a time scale of hours and adjusting synaptic efficacy by scaling up and down. However, the underlying mechanism still remains unclear. Excitatory synaptic efficacy is associated with the size of the dendritic spine, and dendritic spine size fluctuates even after neuronal activity is silenced. These fluctuations could be a non-Hebbian form of synaptic plasticity that serves such a homeostatic function. This study proposed and analyzed a synaptic plasticity model incorporating random fluctuations and soft-bounded Hebbian plasticity at excitatory synapses, and found that the proposed model can prevent excessive changes in neuronal activity by scaling synaptic efficacy up and down. Soft-bounded Hebbian plasticity suppresses strong synapses, thereby scaling synapses down and preventing runaway excitation. Random fluctuations diffuse synaptic efficacy, thereby scaling synapses up and preventing neurons from falling silent. The proposed model acts as a form of homeostatic plasticity, regardless of neuronal activity monitoring.


INTRODUCTION
Synaptic modifications are the fundamental feature for brain development and cognitive processes such as attention, perception, and memory (Bliss and Lømo, 1973;Martin et al., 2000;Whitlock et al., 2006;Nabavi et al., 2014). Synaptic efficacy is adjusted mainly by the process termed as Hebbian plasticity, which depends on the correlation between the activities of the pre-and post-synaptic neurons (Löwel and Singer, 1992;Katz and Shatz, 1996). Traditional Hebbian plasticity and its temporally asymmetric form, spike-timing dependent plasticity (STDP) (Markram et al., 1997;Bi and Poo, 1998;Sjöström et al., 2001), are positive feedback processes that are inherently prone to destabilize the neuronal activity (Rochester et al., 1956;von der Malsburg, 1973;Oja, 1982;Miller and MacKay, 1994;Miller, 1996). Synaptic potentiation due to a correlation between activities of preand post-synaptic neurons further increases the correlation, leading to further potentiation. Once depressed synapse no longer leads to the correlated activities of neurons, leading to further depression and becoming silent. To prevent these instability issues, various constraints on synaptic efficacy have been introduced; limit in the total sum of synaptic efficacy projecting to one neuron (von der Malsburg, 1973;Effenberger et al., 2015), decay in synaptic efficacy (Oja, 1982;Miller and MacKay, 1994), and negative feedback process (Bienenstock et al., 1982;Bear et al., 1987;Miller, 1996). Moreover, so-called "softbounded" STDP models have been proposed (van Rossum et al., 2000;Gütig et al., 2003;Morrison et al., 2007;Gilson and Fukai, 2011), in which depression at a strong synapse is much larger than that at a weak synapse. Soft-bounded STDP models provide an equilibrium at which potentiation and depression are balanced and prevent runaway potentiation or depression. Such dependence on efficacy is supported by evidences from biological studies (Bi and Poo, 1998). However, stability of synaptic efficacy does not always indicate stability of neuronal activity. The ranges of parameter values with which soft-bounded STDP models provide stability of neuronal activity are limited (Morrison et al., 2007;Gilson and Fukai, 2011;Yger and Gilson, 2015). Another negative feedback process, homeostatic plasticity, has also been proposed to maintain neuronal activity within a functional range (van Rossum et al., 2000;Turrigiano, 2008;Watt and Desai, 2010;Zenke et al., 2013;Toyoizumi et al., 2014). It is typically modeled as a proportional or proportional-integral controller that monitors post-synaptic neuronal activity on a time scale of hours, scales synaptic efficacy up or down, and stabilizes postsynaptic neuronal activity. Several neurobiological studies have confirmed the existence of homeostatic plasticity in mammalian cortex and hippocampus Turrigiano et al., 1998;Turrigiano, 1999Turrigiano, , 2008Burrone et al., 2002;Ibata et al., 2008;Keck et al., 2013;Vitureira and Goda, 2013;Félix-Oliveira et al., 2014). After neuronal activity is decreased or silenced by drugs or lesions in sensory organs, synapses are potentiated on average, despite a decrease in correlated activity. Conversely, after neuronal activity is increased by drugs, synapses are depressed on average. Moreover, the normalized distribution of excitatory synaptic efficacy is almost similar before and after the change in neuronal activity. Therefore, this type of homeostatic plasticity is called synaptic scaling or activity-dependent scaling 1 . Synaptic scaling is considered to consist of two distinct underlying mechanisms; scaling up (potentiation against low neuronal activity.) and scaling down (depression against high neuronal activity). Neuronal activity converges to the rate at which scaling up and scaling down are balanced. However, the mechanism underlying synaptic scaling still remains unclear.
Other studies have identified activity-independent (non-Hebbian) excitatory synaptic modifications called intrinsic fluctuations (Yasumatsu et al., 2008;Kasai et al., 2010;Loewenstein et al., 2011;Statman et al., 2014); the size of a postsynaptic element, dendritic spine, fluctuates even after neuronal activity is silenced. Because the efficacy of an excitatory synapse in the mammalian central nervous system is dependent on the corresponding dendritic spine size (Matsuzaki et al., 2001(Matsuzaki et al., , 2004Kasai et al., 2003;Kopec et al., 2006;Wang et al., 2007;Yuste, 2010;Keck et al., 2013), it is possible that these intrinsic fluctuations serve homeostatic plasticity mechanism. However, the intrinsic fluctuations are thought to induce change with zero expected value as opposed to synaptic scaling. This paper proposes an excitatory synaptic plasticity model incorporating intrinsic fluctuations (Yasumatsu et al., 2008) and soft-bounded STDP model (van Rossum et al., 2000), and demonstrates a possible role for these intrinsic fluctuations in scaling of synaptic efficacy to prevent destabilization of neuronal activity. The proposed model demonstrated the robust stability of post-synaptic firing rate in response to change in presynaptic firing rate. The distribution of synaptic efficacy was scaled multiplicatively with varying pre-synaptic firing rate. After neuronal activity was silenced, synapses were scaled up. Intrinsic fluctuations also contributed to synaptic competition (Miller, 1996;Song et al., 2000;van Rossum et al., 2000). These numerical experiments and corresponding theoretical analyses indicate that intrinsic fluctuations achieve scaling up of synapses and soft-bounded STDP model achieves scaling down of synapses. As a result, synaptic plasticity model incorporating intrinsic fluctuations and soft-bounded STDP model acts as a form of homeostatic plasticity without monitoring neuronal activity.

Intrinsic Fluctuations
Recent studies have reported non-Hebbian excitatory synaptic modifications called intrinsic fluctuations (Yasumatsu et al., 2008;Kasai et al., 2010;Loewenstein et al., 2011;Statman et al., 2014). Dendritic spines change size even after neuronal activity is silenced by drugs. Because spine size is strongly related to synaptic efficacy (Matsuzaki et al., 2001(Matsuzaki et al., , 2004Kasai et al., 2003;Kopec et al., 2006;Wang et al., 2007;Yuste, 2010;Keck et al., 2013), these intrinsic fluctuations are thought to influence synaptic efficacy independently from pre-synaptic firing and post-synaptic firing. According to the original studies, intrinsic fluctuations follow the stochastic differential equation as follows: where W t denotes the excitatory synaptic weight W at time t, and B t is the standard Wiener process (i.e., the continuous-time stochastic process with zero mean and unit variance per unit time). Equation (1)

Spike-Timing Dependent Plasticity
Spike-timing dependent plasticity (STDP) is a temporally asymmetric form of Hebbian plasticity that allows both depression and potentiation depending on the temporal relationship between pre-synaptic firing and post-synaptic firing (Markram et al., 1997;Bi and Poo, 1998;Sjöström et al., 2001). When a pre-synaptic firing precedes a post-synaptic firing, the synapse is potentiated, whereas when a pre-synaptic firing follows a post-synaptic firing, the synapse is depressed. The STDP model is commonly expressed as follows: where W is the synaptic weight and W is the magnitude of the change in the synaptic weight W. The synaptic weight W is updated to W + W immediately after whichever occurs later, the pre-synaptic firing (at t pre in depression) or the post-synaptic firing (at t post in potentiation). This is expressed as the temporal difference t = t post − t pre . The functions A + (W) and A − (W) determine the magnitudes W of potentiation and depression, and the parameters τ + and τ − are the time constants of the magnitude W decay. This study employed the multiplicative form of the STDP proposed by van Rossum et al. (2000), which is expressed as where ν p follows a normal distribution with zero mean and variance σ 2 p , i.e., ν p ∼ N (0, σ 2 p ). A strong synapse is easily depressed and further potentiation of the strong synapse is prevented because the amount A − (W) of depression increases with synaptic weight W. Thus, the multiplicative STDP model is also referred to as soft-bounded STDP model (van Rossum et al., 2012). In this study, STDP was assumed to occur not between all possible pairs of pre-and post-synaptic firings but between the pair of a newly generated post-synaptic (pre-synaptic) firing and the latest pre-synaptic (post-synaptic) firing (Morrison et al., 2008;Watt and Desai, 2010). The parameter values were the same as those estimated in the original study by van Rossum et al. (2000): τ + = τ − = 20 ms, c + = 1 pS, c − = 0.003, and σ p = 0.015.

Activity-Dependent Scaling
For comparison, a phenomenological model of activitydependent scaling is introduced (van Rossum et al., 2000). This model monitors the activity of a post-synaptic neuron with a slow-varying sensor a as follows: where δ(·) is the delta function, and t k is the time of the k-th post-synaptic firing. At the state a, the weight W of a synapse connected to the neuron is regulated as follows: where a g (Hz) is the target post-synaptic firing rate. This formulation follows the classical proportional-integral controller. This study employed the same parameter values as those used in the original study, i.e., τ a = 100 s, β = 4 × 10 −5 , and γ = 10 −7 s −1 , except that the target post-synaptic firing rate a g was set to 5 Hz, which is more biologically realistic than the original value of 20 Hz (Mizuseki and Buzsáki, 2013;Buzsáki and Mizuseki, 2014). The features of these models are compared in Table 1.

Dynamics of Neuron and Synapses
The dynamics of the neuron were described by a leaky integrateand-fire model (van Rossum et al., 2000;Izhikevich, 2004): where v is the membrane potential, τ m = 20 ms is its time constant, and v L = −60 mV, v E = 0 mV, and v I = −70 mV are the reversal potentials of leak, excitatory, and inhibitory currents, respectively. Hereafter, the subscripts E and I represent excitatory and inhibitory states, respectively; thus, g E and g I are the conductances of excitatory and inhibitory synaptic currents. R = 100 M is the input resistance. When the membrane potential v reaches the threshold voltage v th = −50 mV, it is reset to v r = −60 mV immediately, and the neuron fires (i.e., generates an action potential). The number of post-synaptic firings per unit time is the post-synaptic firing rate f post . The dynamics of excitatory and inhibitory synaptic conductances are both described as follows: where τ X = 5 ms is the time constant of the conductance. The subscript i is the label of the synapse, W i is the synaptic weight, and s (k) i is the time of the k-th event at synapse i. The weights W of the excitatory synapses were plastic and followed the synaptic plasticity models described above. They are bounded at 0 to prevent a negative weight. The weights W of all inhibitory synapses were fixed at W I = 4000 pS. All the parameter values were the same as those in the original study (van Rossum et al., 2000).
For numerical simulations, all the equations were solved by the forward Euler method with the time step h = 0.1 ms. A single post-synaptic neuron model was constructed that receives inputs from N f = 100 excitatory and N f /4 = 25 inhibitory synapses. The pre-synaptic firing rate was denoted by f pre (Hz). The presynaptic firing rate per time step was p = f pre h. Excitatory presynaptic firings were assumed to be mutually correlated (van Rossum et al., 2000;Bair et al., 2001). This study generated a correlated input as a set of simultaneous excitatory pre-synaptic firings. A correlated group contained N c excitatory synapses. An event within the correlated group followed a Poisson process of rate p c per time step. For each event within the correlated group, m excitatory synapses were randomly chosen from the correlated group and were simultaneously activated. The rate p c was set to pN c /m so that the pre-synaptic firing rate per time

Model
n o m o n it o ri n g o f n e u ro n a l a c ti v it y b io lo g ic a l e v id e n c e p re v e n ti n g lo w a c ti v it y (F ig u re 2 ) p re v e n ti n g h ig h a c ti v it y (F ig u re 2 ) re g u la ti n g w it h v a ry in g fr e q u e n c y (F ig u re 3 A ) s y n a p ti c s c a li n g (F ig u re 3 B ) s ta b il it y o f s y n a p s e s (F ig u re 4 ) re g u la ti n g u n d e r s il e n c e d a c ti v it y (F ig u re 5 ) c o m p e ti ti o n b e tw e e n s y n a p s e s (F ig u re 6 ) "+" denotes "having the property," "−" denotes "having the property partially or with a problem," and a blank denotes "not having the property." step was p. If the random variable X i ∈ {0, 1} represents the occurrence of an event at the excitatory synapse i, the Pearson correlation coefficient between two excitatory synapses i and j in the correlated group is as long as p ≪ 1. This study divided the excitatory synapses into four groups, i.e., N c = 25. Thus, In the case of uncorrelated inputs (i.e., c = 0), the excitatory synapses were activated by homogeneous Poisson processes at f pre (Hz). In both cases, the inhibitory synapses followed homogeneous Poisson processes at f pre (Hz). All the results were obtained from ten trials.

Stability of Neuronal Activity
This section examines the capacity of different STDP models (see Table 1) to maintain neuronal activity in response to a change in synaptic input (f pre ). The excitatory synaptic weights W were initialized to a sufficiently large value, the synapses were activated with a pre-synaptic firing rate f pre = 5 Hz and a correlation c = 0.04, and the post-synaptic neuron accepted the pre-synaptic firings. After a sufficiently long duration, the post-synaptic firing rate f post in response to the pre-synaptic firings reached a steady state. The changes in post-synaptic firing rate f post in response to a sudden decrease in the pre-synaptic firing rate f pre are shown for each model in the left panel of Figure 1. The black line denotes the result obtained from the STDP model with no additional factors, which is referred to as the +NO model. The initial steady-state (baseline) post-synaptic firing rate f post of this model in response to the pre-synaptic firing rate f pre = 5 Hz was very low (< 0.1 Hz). At time t = 0, the pre-synaptic firing rate f pre was decreased to 3 Hz to simulate a condition such as sensory deprivation (Keck et al., 2013). After this decrease, the post-synaptic firing rate f post decreased to approximately zero and never returned to its previous value. To prevent this excessively low post-synaptic firing rate f post , the amplitude c + of potentiation was increased to 1.5 pS in a model referred to hereafter as +LP model, (blue lines in Figure 1). A larger potentiation c + = 1.5 pS induced larger synaptic weights W and a higher baseline postsynaptic firing rate f post in response to the pre-synaptic firing rate f pre = 5 Hz. After decreasing the pre-synaptic firing rate f pre , the post-synaptic firing rate f post temporally fell to zero and quickly started to increase. However, it did not return to baseline. Both models cannot prevent excessively low neuronal activity.
In contrast, the model incorporating intrinsic fluctuations (red line in Figure 1), hereafter denoted as the +IF model, prevented this drastic decrease in the post-synaptic firing rate f post and restored it to baseline, thereby indicating homeostatic plasticity. For comparison, the synaptic plasticity model incorporating activity-dependent scaling and STDP (the +ADS model, purple line in Figure 1) also showed a rapid return to the baseline postsynaptic firing rate f post after an overshoot. Ten synapses were randomly chosen and their trajectories are shown in the lower part of Figure 1. In the case of the +NO model, the synapses were almost unchanged owing to very low post-synaptic firing FIGURE 1 | Changes in post-synaptic firing rate f post following a change in the pre-synaptic firing rate f pre . Homeostatic plasticity is expected to restore the post-synaptic firing rate f post to its previous (baseline) value. (Left panel) The pre-synaptic firing rate f pre is decreased from 5 to 3 Hz. The different colored lines denote results of STDP models with no additional factors (+NO model, black), with a larger potentiation c+ = 1.5 pS (+LP model, blue), with intrinsic fluctuations (+IF model, red), and with activity-dependent scaling model (+ADS model, purple). The post-synaptic firing rates f post of the +NO and +LP models decrease to approximately zero and never return to the previous baseline value. On the contrary, an excessively low post-synaptic firing rate f post is prevented in the +IF and +ADS models. The trajectories of randomly chosen ten synapses are shown in the lower part. In the case of the +NO model, the synapses are almost unchanged owing to very low post-synaptic firing rate f post . In the other cases, some synapses become strong but others become weak; synapses are continuously shuffled even after the post-synaptic firing rate f post converged to a steady state. (Right panel) The pre-synaptic firing rate f pre is increased from 5 to 7 Hz. As is the case with the decreased pre-synaptic firing rate f pre , the +IF and +ADS models restore the post-synaptic firing rate f post to the previous baseline value, whereas the +NO and +LP models do not.
rate f post . In the other cases, some synapses became strong but others became weak; synapses were continuously shuffled even after the post-synaptic firing rate f post converged to a steady state. The pre-synaptic firing rate f pre was then increased from 5 to 7 Hz to simulate increased neuronal activity (right panel of Figure 1). In all four models, the post-synaptic firing rate f post decreased after an initial increase. However, similar to the case of the simulation of decreased pre-synaptic firing rate f pre , the +IF and +ADS models showed rapid restoration of post-synaptic firing rate f post to the baseline value, whereas the +NO and +LP models did not. Thus, the +IF and +ADS models prevented excessive changes in neuronal activity and thereby demonstrated homeostatic plasticity. The trajectories of synapses are also shown in the lower part. In cases with the +IF and +ADS models, synapses were depressed on average. More detailed analyses are provided in the following sections.
In subsequent examinations, the pre-synaptic firing rate f pre and the correlation c of the pre-synaptic firings were widely varied. Figure 2 shows the relationship between pre-synaptic firing rate f pre and resulting post-synaptic firing rates f post at steady state. In the +NO model, the post-synaptic firing rate f post reached a value of ≥ 1 Hz over a range of pre-synaptic firing rate f pre = 0.01-20 Hz only with a high correlation c ≥ 0.08. With a low correlation c < 0.08, the post-synaptic firing rate f post decreased to approximately zero (f post < 0.1 Hz), regardless of the pre-synaptic firing rate f pre . Thus, the soft-bounded STDP model prevented excessively strong synaptic weights W but simultaneously leaded to a post-synaptic firing rate f post of approximately zero because of positive feedback. The +LP model was expected to prevent this excessively low post-synaptic firing rate f post because of a larger potentiation (c + = 1.5 pS). However, again the post-synaptic firing rate f post fell to approximately zero at a pre-synaptic firing rate f pre of < 1 Hz. Moreover, the post-synaptic firing rate f post rose to > 100 Hz at a high pre-synaptic firing rate f pre > 20 Hz or a high correlation c > 0.04. This result indicates that increasing the amplitude c + of potentiation is not enough for preventing almost silent activity and leads to a runaway excitatory as a side effect. As mentioned in the previous studies, the ranges of parameter values with which soft-bounded STDP models provide stability of neuronal activity are strictly limited (Morrison et al., 2007;Gilson and Fukai, 2011;Yger and Gilson, 2015). On the other FIGURE 2 | The post-synaptic firing rates f post in response to varying the pre-synaptic firing rate f pre and the correlation c. The axis of abscissas represents to the pre-synaptic firing rate f pre and the line type refers to the correlation c. The +NO, +LP, +IF, and +ADS models are all STDP models but with no additional factors, a larger potentiation c+ = 1.5 pS, intrinsic fluctuations, or activity-dependent scaling model. At a high pre-synaptic firing rate f pre and a low correlation c, the +NO and +LP models cannot prevent a near-zero post-synaptic firing rate f post . The +LP model also cannot prevent an excessively high post-synaptic firing rate f post . On the other hand, the +IF and +ADS models maintain the post-synaptic firing rate f post within a functional range, indicating that intrinsic fluctuations and activity-dependent scaling model achieve homeostatic plasticity. The gray lines in the third panel denote the results obtained from the +IF model without the noise term ν p , which were almost the same as those obtained from the +IF model with the noise term ν p denoted by the red lines.
hand, the +IF model maintained a post-synaptic firing rate f post within 3-10 Hz over a broad range of pre-synaptic firing rate f pre (0.01-30 Hz) and correlation c (0-0.12) values because the intrinsic fluctuations potentiated synapses with near zero weights (as mentioned in Section 2.1). The post-synaptic firing rate f post varied little with pre-synaptic firing rate f pre but nonetheless increases with correlation c (Figure 2, third panel, red lines). The +IF model has doubly noise terms; intrinsic fluctuations and the noise term ν p ∼ N (0, σ 2 p ) in potentiation and depression of soft-bounded STDP model. The +IF model without the noise term ν p (σ p = 0) was also examined (Figure 2, third panel, gray lines). The results were almost the same as those obtained from the +IF model with the noise term ν p . For comparison, the +ADS model that incorporates both the activity-dependent scaling model and the STDP model, was examined. In this case, regardless of pre-synaptic firing rate f pre and correlation c, the resultant post-synaptic firing rate f post was almost similar to the target post-synaptic firing rate a g = 5 Hz.
This section concludes that intrinsic fluctuations maintain neuronal activity within limits in response to changing conditions (pre-synaptic firing rate and correlation) and can mediate homeostatic plasticity (see Table 1).

Scaling of Synaptic Weights
According to the hypothesis of synaptic scaling Ibata et al., 2008;Keck et al., 2013), neuronal activity is adjusted by scaling the excitatory synaptic weights multiplicatively. To confirm this hypothesis, Figure 3 summarizes the cumulative distributions of synaptic weights W at steady states for the four models. First, the effects of changing pre-synaptic firing rate f pre are shown in Figure 3A.
The +NO, +IF, and +ADS models all yielded lower synaptic weights W with progressive increases in pre-synaptic firing rate f pre ; these results are consistent with the synaptic scaling observed in previous biological experiments Ibata et al., 2008;Keck et al., 2013). Contrary to the previous theoretical studies (van Rossum et al., 2000;Gilson and Fukai, 2011), the +NO model (the STDP model with no additional factors) also achieved synaptic scaling. A pre-synaptic firing always has a chance to induce post-synaptic firing and potentiation. With decreasing pre-synaptic firing rate f pre , the chance that a post-synaptic firing immediately precedes a pre-synaptic firing decreases (as they become sparse); thus, the probability of depression decreases and the synapses are potentiated more easily (see Section 3.2 for details). However, this potentiation is not sufficient for the post-synaptic neuron to elicit action potentials above a certain rate, as shown in Section 3.1.1. In the +LP model with pre-synaptic firing rate f pre = 20 Hz, the synaptic weights W become exceptionally high. According to Figure 2, the post-synaptic firing rate f post diverged from baseline, representing runaway excitation. Therefore, all of these models can adjust synaptic weight, but only the +IF and the +ADS models can adjust it sufficiently to prevent runaway excitation and silence (see Table 1). The +IF model without the noise term ν p (σ p = 0) was also examined ( Figure 3A, third panel, gray lines). The results were also almost the same as those obtained from the +IF model with the noise term ν p .
The synaptic weights W were scaled so that their first and third quartiles were constant with varying pre-synaptic firing rate f pre . The results are summarized in Figure 3B. The shapes of the cumulative distributions of the synaptic weights W were similar in most cases; however, the +NO model with pre-synaptic firing rate f pre = 1 Hz yielded a different shape. This indicates that STDP, which is incorporated into all models examined, achieves synaptic scaling by itself.
Next, the correlation c is varied. It is to be noted that the original biological experiments Ibata et al., 2008;Keck et al., 2013) did not refer to synaptic scaling induced by varying the correlation c because varying correlation c is difficult in biological systems (but not in silico). The results are summarized in Figure 3C. The +NO and +LP models show increased synaptic weights W with higher correlation c, indicating that these models do not provide negative feedback against increasing correlation c. With increase in the correlation c, the +IF model exhibits a decreased number of strong synapses but a greater number of weak synapses. The +ADS model shows depression of all synapses with increasing correlation c. Because synaptic scaling induced by varying the correlation c has not been examined in vivo, this examination cannot draw biological conclusions.

Stability of Strong Synapses
Memory is considered to be stored as synaptic connections and thus strong synapses should stay strong (Fusi et al., 2005;Fusi and Abbott, 2007;Morrison et al., 2007;Gilson and Fukai, 2011). Intrinsic fluctuations however have a risk disturbing retention of memory by depressing strong synapses since they are random fluctuations. However, not only intrinsic fluctuations but also the soft-bounded STDP model shuffles synapses (see the lower part of Figure 1). Effects of intrinsic fluctuations on stability of strong synapses were examined. The pre-synaptic firing rate f pre and the correlation c were set to 5 Hz and c = 0.08, and the post-synaptic firing rates f post reached a steady state. Synapses whose weights are stronger than the 90th percentile of all the synaptic weights of each respective trial at t = 0 were treated as strong. The survival rates, i.e., the rates of the synapses remaining strong from t = 0, are depicted as the solid lines in Figure 4A. The following half-lives were estimated by maximum likelihood; 7.5, 1.9, 4.0, and 4.4 min for the +NO, +LP, +IF, and +ADS models, respectively (see the dotted lines in Figure 4A). In this case, the post-synaptic firing rates f post were 2.02, 16.37, 5.23, and 4.97 Hz for the +NO, +LP, +IF, and +ADS models, respectively (see Figure 2). The estimated half-lives with varying pre-synaptic firing rate f pre are shown in the left panel of Figure 4B; the halflives obtained from the +IF model were almost equal to those from the +ADS model, shorter than those from the +NO model, and longer than those from the +LP model. The increasing order of the half-lives was similar to the decreasing order of postsynaptic firing rates f post (see also Figure 2). Only with a small pre-synaptic firing rate f pre , the +IF model resulted in a shorter half-life despite the lower post-synaptic firing rate f post when compared with the +ADS model. The middle panel of Figure 4B shows the same results except that the axis of abscissas represents the post-synaptic firing rate f post instead of the pre-synaptic firing rate f pre . Owing to homeostatic function, the post-synaptic firing rates f post were varied a little with the +IF and +ADS models. No obvious relationship between the half-life and the post-synaptic firing rate f post was found. The right panel has the product A B FIGURE 4 | Stability of strong excitatory synapses. Synapses whose weights are stronger than the 90th percentile of all the synaptic weights of each respective trial at t = 0 are treated as strong. (A) The solid lines denote the survival rate of the strong synapses in response to the pre-synaptic firing with the rate of f pre = 5 Hz and the correlation of c = 0.08. The dotted lines denote the half-lives estimated by maximum likelihood; 7.5, 1.9, 4.0, and 4.4 min for +NO, +LP, +IF and +ADS models respectively. (B) The estimated half-lives with varying pre-synaptic firing rate f pre . The axes of abscissas represent the pre-synaptic firing rate f pre (left), the post-synaptic firing rate f post (middle), and the product f pre f post of them (right). The numerals in the middle panel indicate the corresponding pre-synaptic firing rates f pre (Hz).
f pre × f post of the pre-and post-synaptic firing rates on the axis of abscissas. The dotted gray line represents 130/(f pre × f post ) (min · Hz −2 ). The half-life was roughly inversely proportional to the product f pre × f post of the pre-and post-synaptic firing rates; the product f pre × f post is also roughly proportional to the occurrence probability of STDP (van Rossum et al., 2000;Gilson and Fukai, 2011) (see also Section 3.2.1). These results suggest that the half-life (i.e., the stability) of the strong synapse is mainly related to the neuronal activity and the resulting occurrence probability of STDP. Intrinsic fluctuations have a slightly negative influence on the stability of strong synapses but it is almost negligible as long as the neuronal activity (i.e., the occurrence probability of STDP) is not too small.

Scaling after Silenced Neuronal Activity
Homeostatic plasticity has been also found during prolonged bath application of the voltage-gated sodium channel antagonist tetrodotoxin (TTX), which silences neuronal activity Ibata et al., 2008). This section highlights the examination of the cumulative distribution of excitatory synaptic weights W in transient states after the pre-synaptic firing rate f pre is reduced to zero. The correlation c was set to 0, and the pre-synaptic firing rate f pre was decreased from 5 to 0 Hz at t = 0. The results are shown in Figure 5. Figure 5A shows the trajectories of randomly chosen ten synapses. The +NO and +LP models did not regulate the synaptic weights after the pre-synaptic firing rate f pre was silenced; the synaptic weights were unchanged. The +IF model shuffled synaptic weights continuously and slowly potentiated synaptic weights on average. The +ADS model potentiated synaptic weights drastically; the trajectories obtained from the +ADS model were clipped for visibility. Figure 5B shows the survival rates of the synapses remaining stronger than the 90th percentile. The +ADS model kept the order of weight since it potentiated synapses multiplicatively. The survival rates obtained from the +NO, +LP, and +ADS models are 1.0. Since intrinsic fluctuations induce change with zero expected value, the +IF model had probabilities for potentiating the weak synapses and depressing the strong synapses; the half-life is 9.7 min. As mentioned in Section 3.1.3, their negative influence on the stability of the strong synapses is no longer negligible. Furthermore, the half-life was obtained in a transient state differently from the half-life in Figure 4, which was obtained in steady state. The synaptic weights are considered to be less stable than those in steady state. Figure 5C shows the cumulative distributions of the excitatory synaptic weights W. The +IF model potentiated the synaptic weights W over several days. The +ADS model potentiated the synaptic weights W, but the post-synaptic firing rate f post never returned to the target post-synaptic firing rate a g (Hz) because the presynaptic firing rate f pre was zero. Consequently, the synaptic weights W became infinite. This result is not consistent with that of the previous biological experiments Turrigiano, 1999Turrigiano, , 2008Burrone et al., 2002;Ibata et al., 2008;Keck et al., 2013;Félix-Oliveira et al., 2014). Hence, based on these results, it can be concluded that the +IF and +ADS models achieve synaptic scaling even after neuronal activity is silenced, but the +ADS model exhibits strange behavior (yields saturated synaptic weights W) under this condition (see A C B FIGURE 5 | Excitatory synaptic weights W in transient states after neuronal activity is silenced. The correlation c is 0, and the pre-synaptic firing rate f pre is decreased from 5 to 0 Hz at t = 0. (A) The trajectories of randomly chosen ten synapses. The +NO and +LP models cannot regulate the synaptic weights after the pre-synaptic firing rate f pre is silenced; the synaptic weights are unchanged. The +IF model shuffles synaptic weights continuously and slowly potentiates synaptic weights on average. The +ADS model potentiates synaptic weights drastically; the trajectories are clipped for visibility. (B) The survival rates of the synapses remaining stronger than the 90th percentile. The +ADS model keeps the order of weight since it potentiates synapses multiplicatively. The survival rates obtained from the +NO, +LP, and +ADS models are 1.0. The +IF model has a probability for depressing the strong synapses; the half-life is 9.7 min. Note that the half-life is obtained in a transient state differently from the half-life in Figure 4, which is obtained in steady state. (C) The cumulative distributions of the excitatory synaptic weights W . The +IF model potentiates the synaptic weights W over several days. The +ADS model potentiates the synaptic weights W, but the post-synaptic firing rate f post never returns to the target post-synaptic firing rate ag (Hz) because the pre-synaptic firing rate f pre is zero. Consequently, the synaptic weights W become infinite. Figure 7A.

Competition
Co-occurring excitatory pre-synaptic firings have a greater chance to induce post-synaptic firing because of summation, thereby potentiating synapses. In contrast to this type of cooperation, synaptic plasticity is also competitive (Miller, 1996;Song et al., 2000;van Rossum et al., 2000;Gütig et al., 2003;Morrison et al., 2007;Gilson and Fukai, 2011); when a group of excitatory synapses has an effect, another group is depressed. Competition between synapses contributes to selectivity of neurons for different input patterns. The traditional STDP models exhibit strong competition often with destabilization of synaptic weights, the soft-bounded STDP models exhibit weak competition, and the activity-dependent scaling model promotes competition. Similar to the activity-dependent scaling model, intrinsic fluctuations are expected to contribute to competition. To examine the competition achieved by synaptic plasticity, two groups A and B of excitatory synapses were chosen from the four groups. Groups A and B had the same pre-synaptic firing rate f pre = 5.0 Hz and had the same or different correlations c A and c B . First, the correlations were set to c A = c B = 0.04. After a sufficiently long duration, the correlation c B was increased to c B = 0.16 at t = 0, as shown in Figure 6A. The resulting changes in the post-synaptic firing rate f post and synaptic weights W are summarized in Figure 6B. In all cases, the post-synaptic firing rate f post increased to a larger value at time t = 0 and the average weightW A of group A synapses started increasing. The response of the average weightW B of group B synapses depended on the plasticity model. In the +NO model, the average weightW B of group B was depressed. In other words, the model resulted in competition between synapses. The +IF and +ADS models also exhibited competition. On the other hand, the average weight W B was almost unchanged in the +LP model. Thus, whether soft-bounded STDP model exhibits competition depends on parameter values (e.g., the amplitude c + of the potentiation). A theoretical analysis is shown in Section 3.2.3.

Under Uncorrelated Activity
In this section, a procedure to obtain a synaptic weight distribution P(W) theoretically is explained, and the mechanisms of homeostatic plasticity and competition are analyzed. The synaptic modifications W yielded by different synaptic plasticity models can be treated as stochastic differential equations as long as W are sufficiently small. A synaptic weight distribution P(W) is obtained using the Fokker-Planck equation (Øksendal, 2003) with the moments M n of the synaptic modifications W, where the moments M n are derived as First, for simplicity, post-synaptic firing is assumed to be unrelated to pre-synaptic firings and to follow a single Poisson process of f post (Hz), which is obtained from the numerical simulations in Section 3.1. This assumption is reasonable when a post-synaptic neuron accepts inputs from numerous synapses or its membrane potential is noisy. In addition, STDP is assumed to occur between all possible pairs of pre-and post-synaptic firings (all-to-all STDP model Morrison et al., 2008;Watt and Desai, 2010) also for simplicity. This approximation is reasonable as long as pre-and post-synaptic neuron firing rates are not excessively high (Gilson and Fukai, 2011). The n-th moments M STDP n (W) of the synaptic modifications yielded by the STDP model are derived as follows (van Rossum et al., 2000;Yasumatsu et al., 2008;Kasai et al., 2010;Gilson and Fukai, 2011;Statman et al., 2014): where P( W) and P( t) are the occurrence probabilities of the potentiation (or depression) with the amount of W and with the temporal difference t. P( t) is equal to the presynaptic firing rate f pre when t > 0 and to the post-synaptic firing rate f post when t < 0 because of the all-to-all STDP model. Because this study employs the multiplicative STDP model, described in Equation (3), the moments M n are as follows: In addition to STDP, the +IF and +ADS models take into account synaptic modifications yielded by intrinsic fluctuations and the activity-dependent scaling model, respectively. The moments M H n of their synaptic modifications are called as the homeostatic terms in this study and expressed as follows (van Rossum et al., 2000;Yasumatsu et al., 2008;Kasai et al., 2010): for other models.
M H 2 (W) = (SW + s) 2 for the +IF model, 0 for other models.
(13) Because of the assumption that the post-synaptic firing follows a single Poisson process of f post , a(t) is clamped to a g and the activity-dependent scaling model does not work. Thus, the moments M n of the synaptic modifications are as follows: The Fokker-Planck equation describes the changes in the synaptic weight distribution P(W, t) as follows: (15) After a sufficiently long duration, the synaptic weight distribution P(W, t) converges to a steady state, which is denoted by P(W). Because the left side is equal to zero, the steady state synaptic weight distribution P(W) is where C is a normalization factor. Figure 7A shows the theoretically-obtained synaptic weight distributions P(W). The colored lines denote the theoretical results and the gray lines denote the corresponding numerical results shown in A B C FIGURE 7 | The theoretically-obtained cumulative distributions of excitatory synaptic weights W. The black and colored lines denote the theoretical results and the gray lines denote the corresponding numerical results shown in Figure 3A. (A) Post-synaptic firing is assumed to be unrelated to pre-synaptic firings and to follow a single Poisson process of f post (Hz) (see Equation 14). Synaptic scaling is observed only in the case of the +IF model. The theoretical results and the numerical results are almost different except for the case of the +IF model. (B) The dependency of post-synaptic firing on pre-synaptic firings is taken into account (see Equation 17). All the models achieve synaptic scaling. By taking into account the dependency terms M Dep n (W), the theoretical results closely resemble the numerical results except for the +LP model with a high pre-synaptic firing rate f pre = 20.0. (C) The cumulative distributions of excitatory synaptic weights W under the competitive situation. The solid lines denote the synaptic weight distribution P(W) when correlated excitatory synapse groups A and B had the same correlation c A = c B = 0.04. The dashed lines denote the synaptic weight distribution P A (W) of correlated group A after its correlation c A is increased to 0.16. The synapses are potentiated. The dotted lines denote the synaptic weight distribution P B (W) of correlated group B after c A is increased to 0.16. When the synapses in the correlated group B are depressed, synaptic scaling is achieved. Figure 3A. They are almost different except for the case of the +IF model. In the absence of intrinsic fluctuations, both the first moment M 1 (W) and the second moment M 2 (W) are directly proportional to the product of the presynaptic firing rate f pre and the post-synaptic firing rate f post . They cancel out in Equation (16), and the synaptic weight distribution P(W) is independent of neuronal activity. In the presence of intrinsic fluctuations, the second moment M 2 (W) is the sum of the activity-dependent term M STDP 2 (W) and the activity-independent term M H 2 (W); thus, the synaptic weight distribution P(W) depends on the product of the pre-synaptic firing rate f pre and the post-synaptic firing rate f post . With increasing pre-synaptic firing rate f pre , the activity-independent term M H 2 (W) is relatively negligible and the synaptic weight distribution P(W) resembles that in the absence of intrinsic fluctuations. With decreasing pre-synaptic firing rate f pre , the activity-independent term M H 2 (W) is dominant and the synaptic weights are widely distributed because of the relatively large diffusion term M 2 (W). This is the mechanism by which the +IF model regulates synaptic weights W against a change in neuronal activity.
In the case of silenced neuronal activity (f pre = 0), the activity-dependent terms M STDP n (W) are zero. Thus, the synaptic weight distributions P(W) obtained from the +NO and +LP models are unchanged. Owing to the activity-independent term M H 2 (W), the +IF and +ADS models mediate synaptic weight. The steady-state synaptic weight distribution P(W) obtained from the +IF model is a power-law distribution Ss (SW+s) 2 with the exponent of 2; it has no well-defined mean weight. The steady-state synaptic weight distribution P(W) obtained from the +ADS model cannot be well-defined; activity-dependent scaling model keeps on potentiating synaptic weights unboundedly. Both the +IF and +ADS models lose stability of synaptic weight distribution in the case of silenced neuronal activity, but the +IF model needs much longer duration than the +ADS model according to the numerical simulations (see Figure 5).

Under Correlated Activity
The pre-synaptic firings have a strong impact on post-synaptic firing. Events at strong synapses or correlated synapses have greater chances to induce post-synaptic firing and potentiation. Next, the dependency of post-synaptic firing on pre-synaptic firings is taken into account. The excitatory synapses are divided into the correlated groups labeled l, each containing N c,l excitatory synapses. A set of m l excitatory synapses in a correlated group l is simultaneously stimulated. A pre-synaptic firing with a weight W in a correlated group l has a chance P fire l (W) to induce post-synaptic firing. Each group l shares the post-synaptic firing rate f post , but has its own synaptic weight distribution P l (W). Equation (14) can be rewritten as At a low pre-synaptic firing rate f pre , the dependency terms M Dep n,l (W) gain influence and potentiate the synapses. At a high pre-synaptic firing rate f pre , the dependency terms M Dep n,l (W) are relatively negligible and the synaptic weight distribution P(W) resembles that when the dependency terms M Dep n (W) are ignored. In other words, a frequent pre-synaptic firing is more likely to induce a depression of other synapses. This is the mechanism by which the soft-bounded STDP model achieves the synaptic scaling. However, synapses potentiated by the dependency terms M Dep n (W) lead to increase in post-synaptic firing rate f post , thereby reducing the influence of the dependency terms M Dep n (W); thus, the soft-bounded STDP model cannot increase post-synaptic firing rate f post sufficiently. This is one reason why the +NO and +LP models cannot prevent excessively low post-synaptic firing rates f post as confirmed in Section 2.1. In contrast, intrinsic fluctuations regulate the synaptic weights W depending not on the dependency terms M Dep n (W) but on the homeostatic terms M H n (W). The homeostatic terms M H n (W) with large values of parameters S and s can keep their influence after the post-synaptic firing rate f post becomes larger; thus, the +IF model prevents the excessively low post-synaptic firing rate f post . In the +ADS model, when the post-synaptic firing rate f post departs from the target a g , the homeostatic terms M H n (W) gain influence and regulate the synaptic weights W to move f post closer to a g .
The result theoretically-obtained from the +LP model with a high pre-synaptic firing rate f pre = 20.0 is not consistent with the numerical result shown in Section 3.1.2 because of a factor not taken into account in the theoretical analysis. When an excitatory pre-synaptic firing with a strong weight is transmitted to the postsynaptic neuron, the membrane potential v gradually approaches the threshold voltage v th . If another excitatory pre-synaptic firing, which is not correlated to the former, is transmitted before the membrane potential v reaches the threshold voltage v th , it hastens the approach. Consequently, the synapses are more likely to be potentiated. This type of correlation between excitatory presynaptic firings gains influence when many synapses become strong and have high firing chances P fire (W); however, it is ignored in this theoretical analysis. For this reason, the +LP model leads to excessively high synaptic weights W. A more detailed analysis is outside the scope of this study.

Competition
The synaptic weight distributions P l (W) of correlated groups l with different correlations c l are also obtained from Equation (17). As is the case with Section 3.1.5, two groups are chosen and labeled A and B. Figure 7C shows the theoretical synaptic weight distributions P A (W) and P B (W) of the correlated groups A and B. According to Equation (20), the dependency terms M Dep n,l (W) of the correlated group l lose their influence and the synapses are depressed when another correlated group k increases its firing chance E[P fire k (W)]. When the correlations are c A = c B = 0.04, the theoretical synaptic weight distributions P A (W) and P B (W) are the same. After the correlation for the correlated group A is changed to c A = 0.16, the firing chance P fire A (W) is increased, and the synapses in the correlated group A are potentiated. The responses of the correlated group B depend on the model. In the cases of the +NO and +LP models, the STDP terms M STDP n,B (W) become dominant because of the increased post-synaptic firing rate f post ; the dependency terms M Dep n,B (W) lose their influence. The correlated group B loses a chance to induce a post-synaptic firing and to be potentiated. In the case of the +IF model, the STDP terms M STDP n,B (W) become dominant, and the homeostatic terms M H n,B (W) lose their influence. In addition to loss of a chance to be potentiated, the correlated group B is scaled down by soft-bounded STDP model because of increase in post-synaptic firing rate f post . In the case of the +ADS model, the activity-dependent scaling model reacts against the increasing post-synaptic firing rate f post and the homeostatic terms M H n,B (W) gain influence. Consequently, the synapses in the correlated group B are depressed. The three models exhibit competition, but their mechanisms differ. If a pre-synaptic firing rate f pre is high, the dependency terms M Dep n,B (W) have lost their influence, and the synaptic weight distribution P B (W) never changes in response to an increase in post-synaptic firing rate f post . This is why the +NO model with the pre-synaptic firing rate f pre = 20 Hz did not exhibit competition in the previous studies (van Rossum et al., 2000;Gilson and Fukai, 2011). However, the +LP model does not exhibit competition in the numerical simulation as shown in Section 3.1.5 for the same reason as that for the excessive potentiation induced of the +LP model shown in the previous section.

Stability of Neuronal Activity
Homeostatic plasticity called synaptic scaling is expected to maintain neuronal activity within a functional range by scaling up and down of synapses (Bienenstock et al., 1982;Turrigiano et al., 1998;Turrigiano, 1999Turrigiano, , 2008Burrone et al., 2002;Ibata et al., 2008;Keck et al., 2013;Vitureira and Goda, 2013;Zenke et al., 2013;Félix-Oliveira et al., 2014;Toyoizumi et al., 2014). Without homeostatic plasticity, the soft-bounded STDP model leaded to a post-synaptic firing rate of excessively low or high (see the left two panels of Figure 2). The soft-bounded STDP model prevents excessive change in synaptic weight; strong synapse is prone to be depressed and weak synapse is prone to be potentiated. Synaptic weights depart from upper and lower bounds and get closer to the equilibrium at which potentiation and depression are balanced with each other. The equilibrium depends on neuronal activity and the soft-bounded STDP model apparently achieves synaptic scaling by itself (see Figure 3A). However, with very low or very high pre-synaptic firing rate, synaptic weights at around the equilibrium lead to excessively high or low post-synaptic neuronal activity. The soft-bounded STDP model achieves at least stability of synaptic weight distribution but does not achieve stability of neuronal activity.
The soft-bounded STDP model incorporating intrinsic fluctuations (+IF model) maintained a post-synaptic firing rate within a certain range over a broad range of pre-synaptic firing rate. According to theoretical analyses, the contribution of intrinsic fluctuations is preventing excessively low neuronal activity by scaling up of synapses. Since the occurrence probability of STDP is roughly proportional to the product of the pre-synaptic and post-synaptic firing rates (van Rossum et al., 2000;Gilson and Fukai, 2011), with decreasing neuronal activity, the soft-bounded STDP model loses its influence and intrinsic fluctuations are relatively dominant; strong synapses are not depressed by the soft-bounded STDP model but rather diffused randomly by intrinsic fluctuations. Therefore, strong synapses stay strong and excessively low neuronal activity is prevented. This is the mechanism for scaling up achieved by intrinsic fluctuations. With increasing neuronal activity, the soft-bounded STDP model gains its influence and intrinsic fluctuations become negligible; strong synapses are depressed and thereby excessively high neuronal activity is prevented. This is the mechanism for scaling down. The soft-bounded STDP model also has the noise term ν p ∼ N (0, σ 2 p ). It is almost negligible when compared with intrinsic fluctuations (see Figures 2, 3A) and also loses its influence with decreasing neuronal activity; the noise term ν p does not contribute to stability of neuronal activity. Therefore, incorporation of intrinsic fluctuations and soft-bounded STDP model achieves scaling up and down of synapses and neuronal activity converges to the equilibrium at which activity-independent scaling up and activity-dependent scaling down are balanced with each other.
Intrinsic fluctuations contribute to competition between synapses but its mechanism differs from the others. When a group of excitatory synapses has an effect, another group reduces correlated activity and loses a chance to being potentiated. This is the mechanism of soft-bounded STDP model for synaptic competition (Song et al., 2000;van Rossum et al., 2000;Morrison et al., 2007). The activity-dependent scaling model reacts against increase in neuronal activity and depresses synapses as is the case with classical negative feedback processes (Bienenstock et al., 1982;Bear et al., 1987;Miller, 1996). On the other hand, intrinsic fluctuations do not react against increase in neuronal activity. Nonetheless increase in neuronal activity promotes soft-bounded STDP model, leading to scaling down of strong synapses which have been potentiated by intrinsic fluctuations. As a result, incorporation of intrinsic fluctuations and soft-bounded STDP model contribute to competition between synapses.
In the case of silenced neuronal activity, the soft-bounded STDP model cannot regulate synapses. Intrinsic fluctuations potentiate synapses slowly because soft-bounded Hebbian plasticity, which normally depresses strong synapses, is blocked. This result is consistent with homeostatic plasticity observed in biological experiments Ibata et al., 2008). After sufficiently long duration, the synaptic weight distribution converges to a power-law distribution with no well-defined mean. Nonetheless synaptic weights are kept within plausible values for several days. The activity-dependent scaling model potentiates synapses drastically, leading implausibly strong synapses within an hour. This is because neuronal activity falls below the target and never returns to the target, no matter how much activitydependent scaling model potentiates synapses. Although both intrinsic fluctuations and the activity-dependent scaling model lose stability of synaptic weight distribution in the case of silenced neuronal activity, intrinsic fluctuations lead to more biologically plausible results. Intrinsic fluctuations derive a power-law distribution with an exponent equal to the degree in synaptic weight of their expression. When expression of intrinsic fluctuations has a term having degree greater than 2 in synaptic weight derives a power-law distribution with a well-defined mean. Examining such intrinsic fluctuations is a future work.

Stability of Synapses
Even when the synaptic weight distribution is stable, it does not imply that the individual synapse is stable. Memory is considered to be stored as synapses connections, stability of synapse contributes to retention of memory. Intrinsic fluctuations however have a risk for disturbing memory retention since they are random fluctuations. This study has shown that intrinsic fluctuations have a slightly negative influence on the stability of strong synapses but it is almost negligible as long as the neuronal activity is not too small. Synapses are shuffled randomly not only by intrinsic fluctuations but also by STDP (see the lower part of Figure 1). A pre-synaptic firing has a chance to induce a post-synaptic firing leading to potentiation of the synapse and depression of another synapse. Frequent pre-and post-synaptic firings induce frequent occurrence of STDP, leading to shuffling of synapses. The influence of intrinsic fluctuations on stability of synapses is much smaller than that of STDP. A more detailed comparison is a future work.
Even after neuronal activity is silenced, intrinsic fluctuations shuffle synapses in contrast to the other models. Their negative influence on the stability of the strong synapses is no longer negligible when the neuronal activity is silenced. This result however does not imply instability of memory; intrinsic fluctuations adapt synapses to the environment that has changed.

Mechanism Underlying Intrinsic Fluctuations
Several molecules and proteins have been considered to be associated with scaling up and down (Turrigiano, 2008). If intrinsic fluctuations are the mechanism underlying scaling up of synapses, they can be mediated by these factors. Another possible hypothesis is that these factors are associated with STDP but not with intrinsic fluctuations, and intrinsic fluctuations are just a noise in post-synaptic structure. Scaling up is not necessarily achieved by promotion of intrinsic fluctuations; it can also be achieved by disturbance of soft-bounded STDP (i.e., scaling down) as shown in Figure 5. In either case, examining effects of these factors on intrinsic fluctuations is a possible future work.

Conclusion
In summary, incorporation of intrinsic fluctuations and softbounded STDP model achieves scaling up and down of synapses and it is one plausible candidate mechanism for homeostatic plasticity called the synaptic scaling.

AUTHOR CONTRIBUTIONS
TM designed research; TM and KU performed research and analyzed data; and TM wrote the paper.

ACKNOWLEDGMENTS
This work was partially funded by the KAKENHI (16K12487 and 26280040) and Kayamori Foundation of Information Science Advancement.