Self-organized chaos through polyhomeostatic optimization D. Markovic and Claudius Gros Institute for Theoretical Physics, Johann Wolfgang Goethe University, Frankfurt am Main, Germany (Dated: June 19, 2018) The goal of polyhomeostatic control is to achieve a certain target distribution of behaviors, in contrast to homeostatic regulation which aims at stabilizing a steady-state dynamical state. We con- sider polyhomeostasis for individual and networks of firing-rate neurons, adapting to achieve target distributions of firing rates maximizing information entropy. We show that any finite polyhomeo- static adaption rate destroys all attractors in Hopfield-like network setups, leading to intermittently bursting behavior and self-organized chaos. The importance of polyhomeostasis to adapting behav- ior in general is discussed. Introduction. – Homeostatic regulation plays a central role in all living, as well as in many technical applica- tions. Biological parameters, like the blood sugar level, the heart beating frequency or the average firing rates of neurons need to be maintained within certain ranges in order to guarantee survival. The same holds in the tech- nical regime for the rotation speed of engines and the velocity of airplanes, to give a few examples. Homeostasic control in the brain goes beyond the reg- ulation of scalar variables like the concentration of pro- teins and ions, involving the functional stability of neu- ral activity both on the individual as well as on a net- work level [1–3]. We use here the term ‘polyhomeosta- sis’ for self-regulating processes aimed at stabilizing a certain target distribution of dynamical behaviors. Poly- homeostasis is an important concept used hitherto mostly implicitly and not yet well studied from the viewpoint of dynamical system theory. Polyhomeostasis is present whenever the goal of the autonomous control is the stabi- lization of a non-trivial distribution of dynamical states, polyhomeostatic control hence generalizes the concept of homeostasis. The behavior of animals on intermediate time scales, to give an example, may be regarded as polyhomeostatic, aiming at optimizing a distribution of qualitatively different rewards, like food, water and pro- tection; animals are not just trying to maximize a single scalar reward quantity. A concept loosely related to poly- homeostasis is homeokinesis, proposed in the context of closed-loop motion learning [4], having the aim to sta- bilize non-trivial but steady-state movements of animals and robots. behavior or self-organized chaos, with both states being globally attracting in their respective phase spaces. Firing-rate distributions. – We consider a discrete-time, rate encoding artificial neuron with input x ∈ $^{[}$−∞ $^{,}$∞ ], output y ∈ [0 , 1] and a transfer function g ( z ), The gain a ( t ) and the threshold − b ( t ) /a ( t ) in (1) are slow variables, their time evolution being determined by polyhomeostatic considerations. Information is encoded in the brain through the firing states of neurons and it is therefore plausible to postu- late [5], that polyhomeostatic adaption for the internal parameters a ( t ) and b ( t ) leads to a distribution p ( y ) for the firing rate striving to encode as much information as possible given the functional form (1) of the transfer function g ( z ). The normalized exponential distribution maximizes the Shannon entropy [7], viz the information content, on the interval y ∈ [0 , 1], for a given expecta- tion value µ . A measure for the closeness of the two probability distributions p ( y ) and p$_{λ}$ ( y ) is given by the Kullback-Leibler divergence [7] which is, through (1), a function of the internal parame- ters a and b . The Kullback-Leibler divergence is strictly positive and vanishes only when the two distributions are identical. By minimizing D$_{λ}$ ( a, b ) with respect to a and b one obtains [6] the stochastic gradient rules which have been called ‘intrinsic plasticities’ [1]. The re- spective learning rates ϵ$_{a}$ and ϵ$_{b}$ are assumed to be small, viz the time evolution of the internal parameters a and b is slow compared to the evolution of both x and y . For Here we study generic properties of dynamical systems governed by polyhomeostatic self-regulation using a pre- viously proposed model [5, 6] for regulating the firing-rate distribution of individual neurons based on information- theoretical principles. We show that polyhomeostatic regulation, aiming at stabilizing a specific target distribu- tion of neural activities gives rise to non-trivial dynami- cal states when recurrent interactions are introduced. We find, in particular, that the introduction of polyhomeo- static control to attractor networks leads to a destruction of all attractors resulting for large networks, as a function of the average firing rate, in either intermittent bursting Δ a = ϵ$_{a}$ ( 1 /a + x Δ ˜ b ) Δ b = ϵ$_{b}$ Δ ˜ b, Δ ˜ b $^{= 1}$− (2 + λ ) y + λy 2 (4) D$_{λ}$ ( a, b ) = ∫ p ( y ) log ( p ( y ) p$_{λ}$ ( y ) ) dy , (3) p$_{λ}$ ( y ) = λ e − λy $^{1}$− e − $_{λ}$, µ = 1 λ e λ − $^{1}$− λ e λ − 1 , (2) y ( t +1) = g ( a ( t ) x ( t )+ b ( t ) , g ( z ) = 1 e − z + 1 . (1) arXiv:1001.0663v2 [cond-mat.dis-nn] 28 May 2010