0.2 0.4 0.6 0.8 1 4 8 12 a stable fixpoints y x a, b 1000 2000 3000 t 0 1 2 3 y(t), b(t), a(t)-4 a(t)-4 b(t) y(t) 2 FIG. 1: (color online) Left: As a function of the average firing rate µ , the region of stability (shaded area) of the fix- point ( y ∗ , b ∗ ), see Eq. (7) of the one-neuron network in the quasistatic limit. Right: A self-coupling neuron with internal parameters a ( t ) and b ( t ). FIG. 2: (color online) The time dependence of y ( t ) (red stars) b ( t ) (solid line) and of a ( t ) − 4 (dashed line) for the balanced one-site problem (5). ϵ a = ϵ b = ϵ = 0 . 01 and µ = 0 . 28; the horizontal dotted lines are guides to the eye. The gain a ( t ) is initially small and the system relaxes, since ϵ ≪ 1, fast to a fixpoint of y = g ( a [ y − 1 / 2]+ b ). Once a ( t ) surpasses a certain threshold, compare Fig. 1, the fixpoint becomes unstable and the system starts to spike spontaneously. any externally given time series x ( t ) for the input, the adaption rules (4) will lead to a distribution of the out- put firing rates y ( t ) as close as possible, given the spec- ification (1) for the transfer function, to an exponential with given mean µ . Attractor relics. – In deriving the stochastic gradient rules (4) it has been assumed that the input x ( t ) is sta- tistically independent of the output y ( t ) (but not vice versa). This is not any more the case when a set of poly- homeostatically adapting neurons are mutually intercon- nected, forming a recurrent network, Here we will show that networks based on polyhomeostatic principles will generically show spontaneous and continuously ongoing activity. In a first step we analyze systematically the smallest possible network, viz the single-site loop, obtained by feeding the output back to the input, see Fig. 1, a setup which has been termed in a different context ‘autapse’ [8]. We use the balanced substitution x → y − 1 / 2 in Eqs. (1) and (4), the complete set of evolution rules for the dynamical variables y ( t ) , a ( t ) and b ( t ) is then with Note, that Δ ˜ b ( t ) in (6) depends on y ( t + 1), and not on y ( t ), as one can easily verify when going through the derivation of the rules (4) for the intrinsic plasticity. The evolution equations (5) are invariant under y ↔ (1 − y ), b ↔ $^{(}$− b ), a ↔ a and λ ↔ $^{(}$− λ ), the later corresponding to the interchange of µ $^{(1}$− µ ). We first consider the quasistatic limit ϵ$_{a}$ ≪ ϵ$_{b}$ , viz a ( t ) ≃ a is approximatively constant. The fixpoint ( y $^{∗}$, b$^{∗}$) in the ( y, b ) plane is then determined by In Fig. 2 the time evolution is illustrated for λ = 3 . 017, which corresponds to µ = 0 . 28, see Eq. (2), and ϵ$_{a}$ = ϵ$_{b}$ = ϵ = 0 . 01. The system remains in the quasistation- ary initial regime until the gain a surpasses a certain threshold. The initial quasistationary fixpoint becomes therefore unstable via the same mechanism discussed an- alytically above, see Eq. (7), for the regime ϵ$_{a}$ → 0, com- pare Fig. 1. The output activity y ( t ) oscillates fast be- tween two transient fixpoints of y = g ( a [ y − 1 / 2] + b ), having a high and a low value respectively. This spiking behavior of the neural activity is driven by spontaneous oscillations in the threshold − b ( t ) /a ( t ), shifting the in- tersection of g ( a [ y − 1 / 2] + b ) with y forth and back. In other words, a finite adaption rate ϵ$_{a}$ for the gain turns the fixpoint attractor ( y $^{∗}$, b$^{∗}$) into an attractor relic and the resulting dynamics becomes non trivial and autonomously self-sustained. This route towards au- tonomous neural dynamics is actually an instance of a more general principle. Starting with an attractor dy- namical system, viz with a system governed by a finite number of stable fixpoint, one can quite generally turn these fixpoints into unstable attractor ruins by coupling locally to slow degree of freedoms [10, 11], in our case the slow local variables are a ( t ) and b ( t ). Interestingly, there are no saddlepoints present in attractor relic networks in general, and in (5) in particular, and hence no unstable heteroclines, as in a proposed alternative route to tran- sient state dynamics via heteroclinic switching. [12, 13] A straightforward linear stability analysis shows, that the fixpoint ( y $^{∗}$, b$^{∗}$) remains stable for small gains a and becomes unstable for large gains, see Fig. 1. We now go beyond the quasistatic approximation and consider a small but finite polyhomeostatic adaption rate ϵ$_{a}$ for the gain. Starting with a small gain a we see, compare Eq. (5), that the gain necessarily grows until it hits the boundary towards instability; for small Δ ˜ b the growths of the gain is a ( t ) ∼ √$_{t}$$_{.}$ y ( t + 1) = g ( a ( t )[ y ( t ) − 1 / 2] + b ( t ) b ( t + 1) = b t ) + ϵ$_{b}$ Δ ˜ b ( t ) (5) a ( t + 1) = a ( t ) + ϵ$_{a}$ ( 1 /a ( t ) + [ y ( t ) − 1 / 2]Δ ˜ b ( t ) Δ ˜ b ( t $^{) = 1}$− (2 + λ ) y ( t + 1) + λy $^{2}$( t + 1) . (6) 0 = λ ( y $^{∗}$) 2 − (2 + λ ) y ∗ + 1 b ∗ = g − $^{1}$( y $^{∗}$) a [ y ∗ − 1 / 2] = log( y $^{∗}$/ $^{(1}$− y $^{∗}$)) a [ y ∗ − 1 / 2] (7)