{'fraction_non_alphanumeric': 0.0394820509856061, 'fraction_numerical': 0.011908202786288225, 'mean_word_length': 4.377681069319242, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 4, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Noise-induced network topologies

Biologically-inspired network models provide a powerful framework for optimization algorithms. One prominent algorithm simulates the networks of filaments built by the unicellular organism Physarum polycephalum. Inspired by nature, we analyse the networks predicted by this dynamics in the presence of stochastic forces simulating a thermal environment. The resulting networks exhibit different topologies as a function of the temperature, which are generally multistable. Remarkably, the system converges to the most robust configuration at non-zero temperatures exhibiting a resonant-like behavior. This configuration is reached with the maximal convergence rate and is not found by the deterministic algorithm. Our results show that stochastic dynamics could qualitatively increase the computational power of optimization algorithms by actively modifying the landscape of the minima corresponding to the sought solutions.

The ability to extract useful information from large data bases has become essential to modern science and technologies. This quest is now central to several fields, ranging from foundational studies, such as in astronomy, for shedding light on the constitution of our universe [1] or in particle physics for efficiently identifying relevant events in high-energy physics experiments [2], to applications, such as the design of efficient power grids [3] and the sustainable exploitation of water supplies [4].
A question lying at the core of these efforts is what are the key ingredients and dynamics at the basis of an efficient search in a generic database. One precious source of insights are living organisms for their capability to extract information from and adapt to a dynamically changing environment [5,6]. A prominent example is Physarum polycephalum, a single-celled organism that is also known as true slime mold [7], see Fig.  1(a). Despite its lack of any form of nervous system, Physarum polycephalum is able to solve difficult computational problems with extreme efficiency like finding the shortest path through a maze [8,9] and creating efficient and fault-tolerant networks [10]. These dynamics are reproduced by coupled nonlinear equations of motion [11]. These equations, in turn, are reference model systems for optimization algorithms successfully applied to several real-world optimization problems [5,12,13].
One relevant aspect of biological systems is the capability to efficiently extract relevant information for their survival in a noisy environment, where parameters fluctuate and the amount and location of food sources can change over time. Fluctuations are at the basis of algorithms such as stimulated annealing and randomized algorithms, where they are known to increase the efficiency of the search over a complex landscape of solutions [14,15]. Yet, noise is also known to have disruptive effects on the dynamics of excitable systems, such as cellular automata and some models of neurons [16]. In nonlinear systems, in fact, stochasticity can induce phenomena such as coherence resonance, stochastic resonance, synchronization, and phase transitions [16][17][18][19]. Some of these features have been reported for simple models of adaptive systems [20][21][22].
In this work we analyse the role of noise on network design. We consider the evolution predicted by a Physarum-inspired model, which reproduces the networks of connections between food sources observed in the experiment by [10]. We analyse the topology and properties of the resulting networks as a function of the amplitude of additive noise, simulating the temperature of an external bath. The network design results from a multicommodity routing optimization [23,24], which is here studied for two demands placed on the grid of Fig.  1(b). The dynamics of the edges connecting two neighbouring nodes is inspired by the food search of Physarum polycephalum and modelled in terms of a transport problem on a circuit with flow-dependent resistivity [11], with the addition of Gaussian noise of variable amplitude (temperature). After convergence, we extract the network over the fluctuating background by means of a filtering procedure as illustrated in 1(c)-(e). As we detail in what follows, the algorithm dynamics exhibits features that are proper to noise-induced resonances at which the resulting topology maximizes robustness and the evolution minimizes the convergence time.
The model. We consider a spatial grid composed by 31 × 31 nodes, where each node u connects to a number of nearest and next-nearest neighbours described by the set E u , as shown in the inset of Fig. 1(b). The network satisfies two demands i = 1, 2, each composed by a source and a sink node, denoted by s i + and s i − , respectively, where the current is injected (+I i ) and extracted (−I i ). The demand i generates a flow I i across the network (i = 1, 2), which consists of the flows Q i u,v across edges connecting nodes (u, v). The flow is conserved at each node u, v∈Eu Q i u,v = 0, with exception for the source and sink where v∈Eu Q i u,v = ±I i . It satisfies the equation where p i u is the potential for the demand i at node u, D u,v is the conductivity of the edge and L u,v is the edge length, which is a constant. The edge dynamics is described by the coupled dynamical variables p i u and D u,v : The potential p i u is determined for each demand i as a function of D u,v (t) by solving a linear system of equations given by Eq. (1) and by the flow conservation at each node (see materials and methods), while the conductivity D u,v (t) obeys the nonlinear equation governed by the activation function f (Q u,v ) in the presence of damping and additive Gaussian noise [20,22]: The activation function simulates the positive feedback of Physarum polycephalum to nutriment flow across a segment. Here, it is the sigmoidal function f (Q u,v ) = x n /(κ n +Q n u,v ) with κ the threshold and Q u,v = i |Q i u,v | the total flow along the edge (u, v). We emphasize that the activation function depends on the total flow. The total flow, in turn, is proportional to the edge conductivities that are the same for all demands. Hence, f (x) gives rise to an effective interaction between demands that favors the sharing of transport routes. The activation is counteracted by dissipation, which is here assumed to occur at the spatially uniform rate γ. Fluctuations in the conductivity due to thermal effects are simulated by the stochastic force ξ(t), whose amplitude is scaled by the parameter α. The force is statistically defined by the average over an ensemble of trajectories: it has no net drift, ξ u,v (t) = 0, and simulates Gaussian white noise, Tuning the parameter α is here equivalent to tuning the temperature of a thermal environment according to T ∝ α 2 [25].
Extracting the network topology. We initialized all conductivities to the value D u,v = 0.5. Figure 1(c) displays a typical network obtained after a sufficiently long simulation time. Edges with a non-zero conductivity D u,v > 0 are drawn with blue lines, whose width is proportional to D u,v > 0. Noise leads to a fluctuating distribution of weak connections. We unveil the network's topology by a filtering procedure consisting of two steps. We first take the time average of the configurations in the regime where the simulation has converged. In this way the fluctuations are levelled out and the noise forms a uniform background, see Fig. 1(d). We then extract the network by applying the disparity filter of Ref. [26] to this timeaveraged configuration (see also Appendix). Figure 1(e) displays the network topology extracted from (c).
Results. Figure 2 summarizes the typical network topologies ordered for increasing noise amplitude. Subplot (A) corresponds to the noiseless case, obtained by integrating Eq. (2) after setting α = 0. In this case the flow satisfying both demands is routed along the vertical connection, since the transport along one edge is bound to a maximal value, the system tends to generate parallel routes. For small α > 0 the individual network backbones generally break the point symmetry (see C-E). For small values of α the network backbones are We perform a systematic characterization of the network designs as a function of the noise amplitude α by analysing an ensemble of 5000 trajectories for each fixed value of α. We measure the backbone of each individual trajectory using the robustness, the transport efficiency, and the cost of the network [28]. The network robustness r is the inverse of the total resistivity R along a flow and provides information on the quality of the connections: it increases by adding paths connecting two nodes, which in turn makes their connection more robust )/I i the effective resistance between the source node s i + and the sink node s i − of each demand i [28]. Note that for calculating the robustness we assume that all edges of the filtered network have the same conductivity per length (here specifically D u,v = 0.5). The transport efficiency σ of the network is given by where d i is the shortest path connecting source and sink nodes s + i and s − i . Finally, the cost of the network c is the total length, found by summing over the segments L u,v where the conductivity is non-zero, with E the set of all edges with non-zero conductivity. The three network measures M = (r, σ, c) are displayed in Fig. 3(a)-(c) as a function of the noise amplitude α. The color code corresponds to the percentage of trajectories at a given value of r, σ, c, respectively, for a fixed noise amplitude α. The labels indicate the corresponding topologies, according to the legenda of Fig. 2. We first observe the existence of multistable solutions at low, but non-zero values of α. One striking feature is that the deterministic configuration A disappears for non-zero values of the noise, indicating that it is unstable against thermal fluctuations. As α is increased, the system jumps to different configurations, undergoing discontinuous transitions between topologies. Remarkably, for a non-zero interval of values α (specifically, within the range 0.001 − 0.002) the dynamics converges to the topology F, which optimizes both robustness and transport efficiency. This interval is characterized by a singlepeaked distribution with a small variance. As α is further increased the distribution broadens again, according to the common expectation that noise gives rise to diffusion over a large number of configurations. We emphasize that the behaviour for noise amplitudes α within the interval 0.001 − 0.002 exhibits the feature of a noise-induced resonance [16]. Remarkably, the associated network topology F has superior robustness and transport efficiency, at the expense of the network cost. At larger values of α we observe the discontinuous transition to network topology (G), which has the same robustness as the deterministic case, a worse transport efficiency, but a definitely lower cost. An important aspect of an optimization problem is the convergence rate to a solution. Figure 4a) and b) display the average convergence rates γ x (x = r, c) and the corresponding variances, respectively, as a function of α. The details of the calculation are reported in the materials and methods. The convergence rates γ x are not monotonous functions of α and exhibit a local maximum at the values of α where the dynamics converges to the network topology (F). Correspondingly, the variances are minimal. This corroborates the conjecture that in the noise range 0.001 − 0.002 we observe a noise-assisted resonant dynamics, leading to the network topology F.
Conclusions. The observation of noise-induced resonances in the network dynamics here reported calls for a systematic theoretical framework defining a cost function for stochastic systems [29]. The close analogy between the equations at the basis of our model and models used for simulating neurons [16] suggests that this study could shed light on neural networks and provide novel concepts for deep learning. In the model we set the length L u,v of the edge u, v equal to unity when the nodes are nearest neighbours and equal to √ 2 when they are connected by a diagonal. We take the exponent of the activation function n = 1.2 and set κ = γ = 1. We impose I i = 0.45 for all demands i. For reference we set the potential at the node u neighboring the source node of demand 1 to the right to p u = 0. The initial state of the simulations has the conductivities of all edges equal to the value D 0 u,v = 0.5. The conductivities are calculated by numerically integrating Eq. (2 ), together with Eq. (1 ). The calculation of the potential p i u at node u and for the demand i is performed by solving a set of linear equations. Let n be the network nodes and x for the noise-free case. Hereby, we averaged over 5000 simulation runs. At the broad resonance at α ∼ 0.002 the network converges to the topology F. b i be the vector determining the constraint, such that it has a value of I i at the source node s i + , −I i at the sink s i − , and 0 otherwise. The vector p i = (p i 1 , ...p i n ) containing all node potentials associated to demand i is found by solving the linear system of equations (23 ) where M is a n × n matrix, M = ADL −1 A T with: (i) D = diag(D e1 , ..., D em ) the m × m diagonal matrix whose eigenvalues are the edge conductivities, (ii) L = diag(L e1 , ..., L em ) the m × m diagonal matrix whose eigenvalues are the edge length, and A the n × m nodearc incidence matrix of the network. In particular, the column (A T ) e has a value of 1 in position u and a value of -1 in position v for all edges e = (u, v). The integration of Eq. (2 ) is performed using stochastic differential equations, that are implemented using the Euler-Maruyama scheme with a step size of ∆t = 0.1γ −1 as outlined in Ref. [30]. The evolution time t end = 250/γ is chosen after testing that each trajectory, namely, each individual evolution of the network, has reached a (meta)-stable configuration. Figures S1 display few trajectories at a fixed value of α.

Appendix B: Steady state and convergence rate
We extend the definition of robustness and cost to a time dependent variable, which we determine on the instantaneous network's backbone, and monitor their dynamics. We identify the steady state as the configurations of the system for which the mean value is constant and the fluctuations are given by the variance set by the noise. To quantify the convergence speed we introduce the quantity γ x for x = c, r, which has the dimensions of a rate and is defined as with the convergence time t x δ = max({t; |x(t )−x(t end )| < δ · x(t end ) ∀ t ∈ [t, t end ]}) and the ensemble average · . In the following we set δ = 0.05, unless otherwise stated. The rates γ x quantify the average convergence rate of the costs and the robustness to the steady state. Furthermore, we define the quantity γ −1 c,r = max(t c δ , t r δ ) which accounts for the combined convergence time of the costs and the robustness. We calculate γ x for different values of the noise strength α by numerically solving the model given by Eq. (1 ) - (2 ). Hereby, we average over 5000 simulation runs for each value of α. In figure 4, the average convergence rates γ x and the standard deviations σ γx are shown as a function of the noise strength α.

Appendix C: Disparity filter
In order to extract the backbone of the network, we take one realization (trajectory) and first average the conductivities D u,v over the time interval [t end − d t , t end ], with d t = 1/γ corresponding to 10 time steps. The time d t is fixed by requiring that over this time the average distribution solely due to noise is stationary and is verified integrating Eq. (2 ) after setting f = 0. After the averaging the effect of thermal fluctuations is leveled to a background value. Furthermore, we introduce a global offset D u,v → D u,v + 0.5 for all edges (u, v). We then apply the disparity filter of Ref. (26 ) using the significance level β = 0.3. The procedure of Ref. (26 ) is implemented as follows. We determine the strength of each node u: s u = v∈Eu D u,v and then normalize the conductivities of the edges that connect the node with its nearest neighbors by p v = D u,v /s u such that v∈Eu p v = 1. We remove all edges whose conductivitises are not statistically significant, i.e. are purely random. As a null hypothesis, it is assumed that the edge conductivities of a certain node of degree k (which can be here either 8, or 5, or 3 depending on the node location within the grid) are produced by a random assignment from a uniform distribution. In order to find the null hypothesis we use the method of induction. For k = 2 edges we have p 1 + p 2 = 1 and p 1 = x where x is a random number in the interval [0, 1]. We divide the interval into infinitesimal steps dx and introduce the probability density ρ(x) such that p 1 = ρ(x)dx. For k = 2 then ρ(x) = 1. For k > 2 we find ρ(x) by solving the nested integral ρ(x)dx = dxk The probability β u,v that the edge (u, v) is compatible with the null hypothesis is given by The disparity filter removes all edges for which it holds β u,v ≥ β with a significance level β ∈ [0, 1] as these edges are not statistically relevant.