{'fraction_non_alphanumeric': 0.03115814226925338, 'fraction_numerical': 0.01038604742308446, 'mean_word_length': 4.886966551326413, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Coherence and clustering in ensembles of neural networks

Large ensembles of globally coupled chaotic neural networks undergo a transition to complete synchronization for high coupling intensities. The onset of this fully coherent behavior is preceded by a regime where clusters of networks with identical activity are spontaneously formed. In these regimes of coherent collective evolution the dynamics of each neural network is still chaotic. These results may be relevant to the study of systems where interaction between elements is able to give rise to coherent complex behavior, such as in cortex activity or in insect societies.

Mutual synchronization may play a significant role in the emergence of coherent collective behavior of different biological populations [1]. Studies of globally coupled nonlinear oscillators show the possibility of spontaneous transitions to a coherent regime characterized by identical states of all elements [2][3][4][5]. Similar transitions, known as complete stochastic synchronization, have been found for globally coupled chaotic logistic maps [6], employed in the description of ecological dynamics [7], and for populations of several coupled chaotic neurons [8]. However, they deem improbable when larger and more complex systems are considered. In contrast to this view, our study reveals that complete mutual synchronization can easily be achieved even in ensembles whose members represent large neural networks with complex individual dynamics. In the emerging coherent regime, the signals generated by all networks in the ensemble are identical. The transition is preceded by clustering, where groups of neural networks with coherent activity patterns are spontaneously formed. Our finding may contribute towards better understanding of coherent activity in the cortex and in biological populations, such as swarms or insect societies.
It can find practical applications in the design of robotic ensembles.
Real neurons are complicated and, in theoretical studies of collective neural activity, greatly simplified models are used [9][10][11]. To investigate synchronization in ensembles of coupled neural networks, we employ the traditional model of McCulloch and Pitts [12] where each neuron is specified by its activity that changes in response to signals arriving from other neurons. A network consists of a set of such elements linked through activatory or inhibitory connections. The dynamics of such a network with arbitrary asymmetric connections is typically characterized by an irregular sequence of complex activity patterns.
In this Letter we consider ensembles made of N identical neural networks each consisting of K neurons. The collective dynamics of an ensemble is described by the following algorithm: At time t + 1, the activity x i k of a neuron k = 1, ..., K belonging to a network i = 1, ..., N is  However, starting from a certain moment, some of the networks in the ensemble begin to generate identical (up to the computer precision) signals, indicating the onset of synchronization in the system. When ε = 0.35 (Fig. 1A), the entire ensemble breaks eventually down into several coherent clusters (the signals generated by networks from the same cluster are displayed here using the same color). At a higher intensity of global coupling (ε = 0.5, Fig. 1B), the activity of all networks in the ensemble becomes eventually coherent. Quite remarkably, the coherent signals are still very complex and apparently chaotic.
The degree of coherence in the ensemble dynamics can be characterized by the dispersion of the activity patterns, defined as Fig. 2 shows, in logarithmic scale, how this property evolves in time in a typical simulation at a fixed intensity ε = 0.5 of global coupling. When global coupling is weak (ε = 0.15, Fig. 3A), the histogram has a single smooth maximum at a typical distance between the activity patterns of noncorrelated networks. Increasing the coupling intensity, we find that above a certain critical point (ε 1 ∼ = 0.17) the nature of the histogram changes. Now, some pairs of networks in the ensemble have exactly the same activity patterns, so that the distance between them is zero. This corresponds to the presence of a peak at d = 0 in Fig. 3B for ε = 0.28. When global coupling is further increased, the number of coherent pairs grows (ε = 0.34, Fig. 3C). Apparently, the coherent networks are already organized into clusters. Indeed, several peaks are clearly seen in this histogram. The peaks are located at pair distances between different coherent clusters.
However, besides the coherent clusters the ensemble still has a number of networks with noncoherent activity. A slight increase of global coupling leads to the emergence of a clear cluster organization (ε = 0.35, Fig. 3D). In this case, every network belongs to one of a few coherent clusters. As the coupling intensity grows, the number of clusters gets smaller, until all networks belong to the same coherent group.
Full coherence is established in the ensemble starting at ε 2 ∼ = 0.4 (this final regime is not shown in Fig. 3). We