{'fraction_non_alphanumeric': 0.03475540881049613, 'fraction_numerical': 0.02319923538100617, 'mean_word_length': 4.773513920240783, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Learning excitatory-inhibitory neuronal assemblies in recurrent networks

Understanding the connectivity observed in the brain and how it emerges from local plasticity rules is a grand challenge in modern neuroscience. In the primary visual cortex (V1) of mice, synapses between excitatory pyramidal neurons and inhibitory parvalbumin-expressing (PV) interneurons tend to be stronger for neurons that respond to similar stimulus features, although these neurons are not topographically arranged according to their stimulus preference. The presence of such excitatory-inhibitory (E/I) neuronal assemblies indicates a stimulus-specific form of feedback inhibition. Here, we show that activity-dependent synaptic plasticity on input and output synapses of PV interneurons generates a circuit structure that is consistent with mouse V1. Computational modeling reveals that both forms of plasticity must act in synergy to form the observed E/I assemblies. Once established, these assemblies produce a stimulus-specific competition between pyramidal neurons. Our model suggests that activity-dependent plasticity can refine inhibitory circuits to actively shape cortical computations.


19
With the advent of modern optogenetics, the functional role of inhibitory interneurons has developed 20 into one of the central topics of systems neuroscience [Fishell and Kepecs, 2019]. Aside from the 21 classical perspective that inhibition serves to stabilize recurrent excitatory feedback loops in neuronal 22 circuits [van Vreeswijk and Sompolinsky, 1996, Brunel, 2000, Murphy and Miller, 2009 Figure 1: Homeostatic plasticity in input and output synapses of interneurons drives the formation of E/I assemblies. a. Emergence of E/I assemblies comprised of pyramidal neurons (triangles) and parvalbumin-expressing interneurons (circles) in circuits without feature topography. b. Network architecture and stimulus tuning of external inputs to pyramidal (Pyr) cells. Continued on following page.  (RS). Numbers correspond to points marked in (e). e. Relationship of synaptic efficacies of output (left) and input connections (centre) of PV interneurons with response similarity. Relationship of input and output efficacies (right). Black lines are obtained via linear regression. Reported r and associated p-value are Pearson's correlation. f. Stimulus tuning of excitatory and inhibitory currents onto an example Pyr cell, before and after learning. For simplicity, currents are shown for spatial and temporal frequency only, averaged across all orientations. g. Angle between the weight update and the gradient rule while following the local approximation for input (top) and output (bottom) connections of PV interneurons. Time course for first 4% of simulation (left) and final distribution (right) shown. Black lines are lowpass filtered time courses. 2018]. We wondered whether a stimulus-specific form of homeostasis can also generate stimulus-specific 87 feedback inhibition by forming E/I assemblies. To that end, we derive synaptic plasticity rules for ex-88 citatory input and inhibitory output connections of PV interneurons that are homeostatic for the 89 excitatory population (see Materials & Methods). A stimulus-specific homeostatic control can be seen 90 as a "trivial" supervised learning task, in which the objective is that all pyramidal neurons should learn 91 to fire at a given target rate ρ 0 for all stimuli. Hence, a gradient-based optimisation would effectively 92 require a backpropagation of error [Rumelhart et al., 1985] through time [BPTT; Werbos, 1990]. 93 Because backpropagation rules rely on non-local information that might not be available to the 94 respective synapses, their biological plausibility is currently debated [Lillicrap et al., 2020, Sacramento 95 et al., 2018, Guerguiev et al., 2017, Whittington and Bogacz, 2019, Bellec et al., 2020. However, 96 a local approximation of the full BPTT update can be obtained under the following assumptions: 97 First, we assume that the sensory input to the network changes on a time scale that is slower than 98 the intrinsic time scales in the network. This eliminates the necessity of backpropagating information 99 through time, albeit still through the synapses in the network. This assumption results in what we 100 call the "gradient-based" rules (Eq. 15 in the Supplementary Materials), which are spatially non-local. 101 Second, we assume that synaptic interactions in the network are sufficiently weak that higher-order activity r I and the signed deviation of total postsynaptic pyramidal cell input h E from the homeostatic target: In contrast, the PV input synapses W I←E are changed such that the total excitatory drive I E, rec i from the Pyr population to each interneuron is close to some target value I 0 (Supp. Fig. S1a, right): Both synapse types are subject to a weak weight decay, to avoid the redundancy that a multiplicative 108 rescaling of input synapses can be compensated by a rescaling of the output synapses.
and output connections are highly correlated (Fig. 1e, right). The stimulus tuning of the inhibitory  Finally, the optimal gradient rules produce very similar results to the local approximations (Supp. 130 Fig. S4). Over the course of learning, the weight updates by the approximate rules align to the updates 131 that would result from the gradient rules over (Fig. 1g, Supp. Fig. S3), presumably by a mechanism 132 akin to feedback alignment [Lillicrap et al., 2016, Akrout et al., 2019. when those are log-normally distributed.

147
When we knock-out output plasticity but keep input plasticity intact, the network fails to develop 148 E/I assemblies and a stimulus-specific E/I balance. While there is highly significant change in the 149 distribution of PV interneuron stimulus selectivity (Mann-Whitney U test, U = 1207, p < 10 −4 ), 150 the effect is much stronger when output plasticity is also present (Fig. 2a,  Dashed line marks threshold of high significance (p< 0.01). f. Fraction of samples with highly significant positive correlation before plasticity, after plasticity in both input and output connections and for knock-out (KO) of plasticity in PV output connections (based on 10,000 random samples of 100 synaptic connections).
(2% of samples). Notably, we find a correlation between output weights and response similarity in 0.0% 163 of samples (Fig. 2f). Finally, for an experimentally realistic sample size of n = 100, the probability of 164 a correlation coefficient equal or higher than that observed by Znamenskiy et al. [2018] is 0.0% for the 165 correlation between output weights and response similarity (r = 0.55), and 0.0% for the correlation 166 between input and output synapses (r = 0.52).

167
The non-local gradient rule for the PV input synapses alone also does not permit the formation of 168 E/I assemblies (Supp. Fig. S4a). While the selectivity of interneurons increases more than for the local 169 approximation (Supp. Fig. S4b), feedback inhibition still remains untuned in the absence of output 170 plasticity (Supp. Fig. S4c,d). 171 We therefore conclude that input plasticity alone is insufficient to generate the synaptic microstruc-172 ture observed in mouse V1.

173
Pyr → PV plasticity is required for assembly formation

174
When we knock out input plasticity but keep output plasticity intact, we again observe no formation 175 of E/I assemblies. This remains true even when using the gradient-based rule (Supp. Fig. S4). The 176 underlying reason is that input weights remain random. Interneurons collect excitation from many Pyr 177 neurons with different preferences, and absent plasticity on their input synapses, they maintain their 178 initial poor stimulus selectivity ( Fig. 3a-c). Because of the poor stimulus tuning of the interneurons, 179 output plasticity cannot generate stimulus-specific inhibitory inputs to the Pyr neurons ( Fig. 3d).

180
Across the whole population, the similarity of excitatory and inhibitory currents onto Pyr neurons 181 remains low (Supp. Fig. S2b,c).

182
Note that interneurons still possess a weak, but consistent stimulus tuning that arises from random Arrows indicate median. c. Violin plots of inhibitory stimulus selectivity before plasticity, after learning with plasticity in both input and output connections of PV interneurons and for knock-out of plasticity in PV input connections. d. Stimulus tuning of excitatory and inhibitory currents in a Pyr cell before and after learning. Dimensions correspond to spatial and temporal frequency of the stimuli averaged across all orientations. e. Fraction of samples with highly significant (p < 0.01) positive correlation before plasticity, after plasticity in both input and output connections, and for knockout (KO) of plasticity in PV input connections (based on 10,000 random samples of 100 synaptic connections). PV input and output plasticity in place to learn E/I assemblies. Second, both before and after learning,  S6b). This confirms that a "blanket of inhibition" does not account for 221 feature-specific suppression between excitatory neurons [Sadeh and Clopath, 2020].

222
To better understand this behaviour, we use the Pyr-Pyr receptive field correlations to compute 223 the coefficient of determination for all pairs (R 2 , which quantifies how well the receptive field of one 224 Pyr neuron predicts that of another). Learning changes the correlative structure in the network (Supp.   Chang et al., 2010, Rutherford et al., 1998, Lamsa et al., 2007. In summary, we conjecture 305 that PV interneurons and their excitatory inputs have the necessary prerequisites to implement the 306 suggested local Pyr → PV plasticity rule.

307
If excitatory inputs to Pyr neurons are much larger than required to reach the target, the homeo-308 static objective of bringing net currents to that target effectively requires a balance of excitation and 309 inhibition on a stimulus-by-stimulus basis, with a small overshoot of excitation (or, in spiking networks, 310 membrane potential fluctuations) that allows Pyr neurons to fire at the target rate. We speculate that 311 E/I assemblies could be learned not only from the homeostatic objective used here, but by any other 312 objective that enforces a positive correlation of the stimulus tuning of excitatory and inhibitory inputs 313 to neurons in the circuit.

314
We expect that the rules we suggest here are only one set of many that can establish E/I assemblies.

315
Given that the role of the input plasticity in the interneurons is the formation of a stimulus specificity, 316 it is tempting to assume that this could equally well be achieved by classical forms of plasticity like the 317 Bienenstock-Cooper-Munro (BCM) rule [Bienenstock et al., 1982], which is commonly used in models of receptive field formation. However, in our hands, the combination of BCM plasticity in Pyr → PV 319 synapses with homeostatic inhibitory plasticity in the PV → Pyr synapses showed complex dynamics, 320 an analysis of which is beyond the scope of this article. In particular, this combination of rules often 321 did not converge to a steady state, probably for the following reason. BCM rules tend to make the  pect that in juvenile mice single-cell perturbations would not cause feature-specific suppression but 382 amplification due to excitatory recurrence and unspecific feedback inhibition.

Materials & Methods
We use custom software to simulate a rate-based recurrent network model containing N E = 512 386 excitatory and N I = 64 inhibitory neurons. The activation of the neurons follows Wilson-Cowan 387 dynamics: The cropping threshold C is chosen such that the overall connection among the excitatory neurons 400 probability is 0.6. The remaining synaptic connections (E→I, I→E, I→I) are initially random, with a 401 connection probability p = 0.6 and log-normal weights. For parameters please refer to Table 1 The PV → Pyr and Pyr → PV synapses follow plasticity rules that aim to minimize the deviation of 408 the excitatory activations from a target rate ρ 0 (ρ 0 = 1 Hz): where · s denotes the average over all stimuli. When plastic, synaptic weights change according to After every update of the Pyr → PV matrix, the incoming weights for each PV interneuron are to improve optimisation performance. 418 We used a standard reparameterization method to ensure the sign constraints of an E/I network.

419
Moreover, all weights are subject to a small weight-dependent decay term, which aids to keep the The same measure is used for the similarity of synaptic currents onto excitatory neurons in Supp. ρ 0 1 Hz Homeostatic plasticity target.
10 −9 synapses in the network. The rules that we investigate here tend to produce bimodal distributions of 457 weights, with the lower mode well below this threshold (Supp. Fig. S7).

458
The stimulus selectivity of the neurons is measured by the skewness of their response distribution 459 across all stimuli: wherer i = r i (s) s . Both the response similarity Eq. (5) and the stimulus selectivity Eq. (6) are 461 adapted from Znamenskiy et al. [2018].

462
Finally, the angle θ between the gradient G from Eq. (15) and its approximation A from Eq. (4) is 463 given by: The general framework we follow to derive homeostatic rules is to minimise the mean squared deviation 676 of individual excitatory (Pyr) neuron activations from a target for all stimuli. More specifically, we 677 perform gradient descent on the following objective function: Note that the activations h E are given by the difference between the excitatory and the inhibitory 679 inputs to the excitatory neurons. Our approach can hence be interpreted as supervised learning of 680 the inhibitory circuitry, with the goal of minimising the mean squared loss between the inhibitory and 681 the excitatory inputs (plus the constant target ρ 0 ). In this sense, the derived gradient rules aim to 682 generate the best possible E/I balance across stimuli that is possible with the circuitry at hand.

683
For reasons of readability, we will first simply state the derived rules. The details of their derivation 684 can be found in the following section.

685
The sign constraints in excitatory-inhibitory networks require all synaptic weights to remain posi-686 tive. To ensure this, we reparameterised all plastic weights of the network by a strictly positive soft-plus 687 function W = s + (V ) = α −1 ln (1 + exp αV ) and optimised the weight parameter V by gradient descent.

688
In summary, the derived learning rules for the synaptic weight parameters between excitatory 689 neuron j and inhibitory interneuron i are given by Please note that we added a small weight decay to both learning rules. The purpose of this decay term changes. This assumption significantly simplifies the derivation of the gradient.

709
The goal is to minimise the total deviation of the excitatory activations h E from the homeostatic 710 target value ρ 0 . To this end, we calculate the gradient of the objective function in Eq.
(3) with respect 711 to a given synaptic weight parameter We therefore need the gradient of the activations h E of excitatory cells with respect to a parameter v.

713
In the steady state, the activations are given by The gradient of the activations h E is therefore given by the following implicit condition: where we introduced the diagonal matrices D Eq. (11) requires the gradient ∂h I ∂v of the inhibitory activations with respect to the parameter v, which can be calculated by a similar approach Introducing the effective interaction matrix M := I + W I←I D I among the interneurons (I being the identity matrix) allows to solve for the gradient of h I : Inserting this expression into Eq. (11) yields excitatory neurons yields an explicit expression for the gradient of h E : To obtain gradients with respect to a particular network parameter, we simply substitute the chosen and for the parameters V E←I ij of the output synapses from the interneurons we get By inserting these expressions into Eq. (9) and dropping the average, we obtain online learning rules for the input and output synapses of the interneurons: Note that the same approach also yields learning rules for the threshold and the gain of the transfer function of the inhibitory interneurons, if those are parameters of the system. Although we did not use such intrinsic plasticity rules, we include them here for the interested reader. We assumed a threshold linear transfer function of the interneurons: where g i is the gain of the neuronal transfer function and θ i a firing threshold. While the firing threshold can become negative, gain is reparameterised via the strictly positive soft-plus g i = s + (v g i ).
The gradient-based learning rule for the firing thresholds θ i of the interneurons is given by and the corresponding learning rule for the interneuron gain g i is Approximating the gradient rules

725
In the gradient-based rules derived in the previous section, the W −1 and M −1 terms account for the 726 fact that a change in a given synaptic connections percolates through the network. As a result, the 727 learning rules are highly nonlocal and hard to implement in a biologically plausible way. To resolve 728 this challenge, we begin by noting that which holds if ||Ŵ|| < 1.Ŵ is a matrix that depends on the synaptic weights in the network. A similar relation holds for M −1 . Since those matrices are contained in Eq. (15a), we substitute the equivalent sums into the relevant sub-expression and truncate the geometric series after the 0-th order, as in are full rank). One way of assuring this is a sufficiently positive diagonal of this matrix product, i.e.

752
While such a positive correlation is not necessarily present for all learning tasks or network models, 753 we speculate that it will be for the task of learning an E/I balance in networks that obey Dale's law.

754
The same logic of using a 0-th order approximation of W −1 that neglects higher order interactions 755 is employed to recover the inhibitory synaptic plasticity rule of Vogels et al. [2011] from Eq. (15b).

756
Overall, the local approximation of the learning rule relies on three assumptions: Slowly varying 757 inputs, weak synaptic weights and alignment of input and output synapses of the interneurons. These 758 assumptions clearly limit the applicability of the learning rules for other learning tasks. In particular, 759 the learning rules will not allow the network to learn temporal sequences.  Figure S2: Synaptic currents onto Pyr neurons. a. Excitatory (red) and inhibitory (blue) synaptic currents onto a random selection of Pyr neurons, as a function of temporal and spatial stimulus frequency (averaged over all orientations), when both incoming and outgoing PV synapses are plastic. b. The network-averaged excitatory (first row) and inhibitory (second row) synaptic currents onto Pyr neurons, both centred according to the peak excitatory current before averaging. After averaging their difference is taken (third row), and a slice is plotted (bottom row). When both plasticities are present, currents are well-balanced across stimuli with a modest excitatory bias for preferred stimuli. c. Quality of E-I current co-tuning for every Pyr in the network quantified by the distribution of their cosine similarities. Only when both plasticities are present do most Pyr neurons receive well co-tuned E-I synaptic currents. Figure S3: Both in-and output synapses must be plastic for feedback alignment to occur. In a network with both local rules (left column), the update to Pyr → PV synapses rapidly align to the gradient (i.e. when the angle between the approximate update and the gradient is below π/2; bottom left). While updates to the Pyr → PV weights occasionally point away from the gradient, 79% of samples are below π/2. For the knock-out (KO) experiments (right column), output plasticity closely follows the PV → Pyr gradient even if input plasticity is absent (upper right). In contrast, if output (PV → Pyr) plasticity is absent the approximate Pyr → PV rule does not follow the gradient (lower right). Figure S4: Gradient rules also require plasticity of both in-and output synapses of PV interneurons. a. In a network learning with the derived gradient rules of Eq. (15), significant correlations are reliably detected between response similarity (RS) and excitatory weights (red bars), RS and inhibitory weights (blue bars), and excitatory & inhibitory weights for reciprocally connected Pyr-PV cell pairs (black bars) only if both synapse types are plastic. b. Interneurons fail to develop stimulus selectivity if their input weights do not change according to the gradient rule of Eq. (15a). c. Synaptic currents onto Pyr neurons only develop reliable, strong excitatory-inhibitory (E/I) co-tuning if both in-and output synapses are updated using the gradient rules. Currents are averaged across all Pyr neurons after centering according to the neuron's preferred stimulus. The bottom row is a slice through the difference (third row) of the average excitatory (first row) and inhibitory currents (second row). d. Violin plot of the distribution of E/I synaptic current similarity values for all Pyr neurons in the network (see Methods). Figure S5: Correlation between weights and response similarity. Scatter plots containing every synapse in networks without PV → Pyr plasticity (top), or without Pyr → PV plasticity (bottom). Pearson correlation is always highly significant, though sometimes weak. Figure S6: In-and output plasticity together change correlations between pyramidal (Pyr) neurons, while plasticity knock-out (KO) eliminates feature competition. a. Receptivefield correlations (Pearson) between Pyr neurons, before (top) and after (bottom) learning with both PV → Pyr and Pyr → PV synaptic plasticity. b. The effect of perturbing a Pyr neuron on the response of other Pyr neurons (to random stimuli) as a function of their receptive-field correlation (see Materials & Methods). On their own, both Pyr → PV and PV → Pyr plasticity have little effect on the feature amplification observed prior to learning. c. Despite the absence of feature competition on average in the KO networks, the total strength of Pyr → PV synapses from a given Pyr neuron is still predictive of its influence on the rest of the network: The stronger its total weight, the more likely a Pyr is to suppressing the response of other Pyr neurons. Figure S7: Some networks contain experimentally undetectable weights. a. Plots of 2D histograms for PV → Pyr (top) and Pyr → PV (bottom) weight versus response similarity (RS), in different networks trained with the local plasticity rules (columns). White lines indicate the threshold of experimental detectability. Any weight < 10 −4 is not included when computing Pearson's correlation between RS and synaptic weight, or weight-weight correlations. b. Same plots as (a), but for networks trained with the gradient rules.