{'fraction_non_alphanumeric': 0.04303977805271248, 'fraction_numerical': 0.009597720541371424, 'mean_word_length': 4.753666954270924, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Reveal flocking of birds flying in fog by machine learning

We study the first-order flocking transition of birds flying in low-visibility conditions by employing three different representative types of neural network (NN) based machine learning architectures that are trained via either an unsupervised learning approach called"learning by confusion"or a widely used supervised learning approach. We find that after the training via either the unsupervised learning approach or the supervised learning one, all of these three different representative types of NNs, namely, the fully-connected NN, the convolutional NN, and the residual NN, are able to successfully identify the first-order flocking transition point of this nonequilibrium many-body system. This indicates that NN based machine learning can be employed as a promising generic tool to investigate rich physics in scenarios associated to first-order phase transitions and nonequilibrium many-body systems.


I. INTRODUCTION
Recent enormous development in machine learning techniques [1][2][3] has stimulated considerable interests and efforts in applying these powerful techniques in physics research. A case in point is utilizing the remarkable abilities of these techniques in recognizing, classifying, and characterizing complex sets of data to facilitate one of the central tasks in modern physics research, namely, identification and classification of collective phenomena and phases of physical systems with huge number of degrees of freedom. Over the past few years, various machine learning techniques have successfully been applied in classifying phases of different physical systems and models ranging from prototypical classical Ising-type models [4,5], over strongly correlated fermions [6,7], to Kitaev chain, disordered quantum spin systems [8][9][10][11], etc. (see Ref. [12] for a more through review on related topics).
In this context, physics research in two scenarios that worth systematically applying machine learning techniques, however, receive relatively less attention so far. One scenario is associated to systems manifesting firstorder phase transitions. As it is well known, in sharp contrast to the continuous phase transition, there is no diverging correlation length scale associated to the firstorder phase transition. On the one hand, this protects their rich physics at different length scales from being washed out by the diverging correlation length scale, while on the other hand, this also makes their physics difficult to be studied by powerful tools such as renormalization group [13]. The other scenario is associated to nonequilibrium (NEQ) many-body systems where detailed balance is generally absent. Comparing to their equilibrium counterparts, on the one hand, the absence of detailed balance naturally gives rise to much richer physics, however, on the other hand, it also makes devel-oping the general framework and efficient tools for understanding their rich NEQ physics much harder [14]. In these regards, this thus poses the intriguing question whether tools from machine learning can be generally applied to investigate the rich physics in these two scenarios.
As a first step to address this question, here we employ three representative types of neural network (NN) based machine learning architectures to investigate the physics of a prototypical NEQ many-body system that manifests a first-order phase transition (cf. Fig. 1). More specifically, we utilize machine learning to investigate the flocking behavior of self-propelled particles under influences from environmental fluctuations, for instance, birds flying in low-visibility conditions like foggy weather. This is an intrinsic NEQ system [15,16] that assumes a firstorder transition between the flocking and the disordered phase [17,18], and is usually described by a modified version of the well-known Vicsek model [19] with extrinsic noise that accounts for the influences from the environment [17,18]. Here, our first goal is to study whether, without "special fine tuning" on the inner architectures of these representative NNs, they can be directly applied to classify system's NEQ phases and identify the associated first-order phase transition. The second goal is to study how different inner structures of these NNs influence their final predictions concerning the first-order flocking transition, which is crucial for further applying these techniques to related physics research. We achieve our goals by training the NNs with two different types of approaches, namely, an unsupervised learning approach called "learning by confusion" proposed in Ref. [8], and a supervised learning approach widely applied in different physical systems [4,6,7,10]. With the different NNs after training, we are able to extract the critical extrinsic noise level of the first-order flocking transition by using any one of them trained via either the unsupervised learning approach (cf. Fig. 2) or the supervised learning one [cf. Figs. 3(a, b, c)]. We further find the predicated critical noise levels by different NNs trained via different approaches agree well with each other, and with the estimation from direct numerical simulations. These systematic studies thus suggest that NN based machine learning is a class of promising generic tools to investigate rich physics in scenarios associated to first-order phase transitions and NEQ systems.
The paper is organized as follows: In Sec. II we first introduce the physical system and model under study, then we briefly present the data generation and the three representative NN based machine learning architectures employed in this work, namely, the fully-connected NN, the convolutional NN, and the residual NN. In Sec. III we discuss the results from different NNs trained via the unsupervised learning approach and the supervised learning approach, respectively. We conclude and give an outlook in Sec. IV.

II. SYSTEM AND METHOD
A. Birds flying in fog and Vicsek model with extrinsic noise The physical system under consideration consists of N self-propelled particles in the two-dimensional (2D) space of linear size L under influences from environmental fluctuations, for instance, birds flying in low-visibility conditions such as foggy weather. It can be modeled by a set of stochastic discrete-time dynamical equations [17,18], which corresponds to a modified version of the well-known Vicsek model [19] with extrinsic noise, whose explicit form reads where r j (t), v j (t), and θ j (t) are the position, the velocity and the direction angle of the velocity of the jth bird at time t (from now on we directly refer self-propelled particles as birds), respectively. Here, all birds are assumed to fly at speeds with the same constant magnitude v 0 . ∆t is the discrete time step and one can choose ∆t = 1 without loss of generality. U j is the spherical neighborhood of the jth bird with radius r 0 centered at r j . Influences from environmental fluctuations are described by the extrinsic noise term ηe iξj (t) appearing in Eq. (2), with ξ j (t) being a random variable uniformly distributed within the interval [−π, π] and η being the noise level. A key feature of this system is that, for the fixed density ρ ≡ N/L 2 of birds, the change in the noise level can drive a firstorder phase transition [17,18] between the flocking phase and the disordered phase [cf. insets of Fig. 1(a)], which is characterized by the jump in the magnitude of the average normalized velocity v a = | N j=1 v j |/N v 0 , as shown for instance in Fig. 1(a).
In the following, we focus on the case with N = 2048, ρ = 2.0, v 0 = 0.5, r 0 = 1, and shall use NN based ma-chine learning techniques to investigate the first-order flocking transition of this system and particularly predict the critical noise level associated to this transition.

B. Data generation and NN architectures
In order to utilize NNs to classify different phases of the systems and consequentially investigate its first-order flocking transition, one needs first to provide sufficient amount of data concerning the system to the NNs. Here, the data to be directly provided to and processed by different NNs are images of steady state configurations of the system at different noise levels, which are obtained from numerical simulations on the stochastic dynamical equations (1, 2, 3) with periodic boundary condition imposed. Specifically, the steady state configurations, namely, the position and velocity distributions at different time point t in the steady state, are transformed into images of 3 × 224 × 224 pixels by using the directions and the positions of N short arrows to denote the directions of velocities and the positions of the birds, respectively. The image size "3 × 224 × 224" is the standard choice for the well-established residual NN [24], whose structure is briefly presented at the end of this subsection. These images form the data set for performing machine learning and are divided into three categories, namely, the "training set", the "validation set", and the "test set".
In this work, we employ three different types of NNs to process the data, i.e., the fully-connected NN, the convolutional NN, and the residual NN [cf. Figs. 1 The learnable parameters (to be specified in the following discussion) of these NNs are optimized by using the Adam method [25], which is an adaptive variant of the "stochastic gradient descent" algorithm. Before going into the detailed discussion on our results of machine learning, we briefly present in the following the major architectures of the three different types of NNs and how data are processed in each of them (for more thorough discussions of related technical details, we refer the reader to Refs. [2,3]).

Fully-connected NN
One type of NNs employed in this work is the fullyconnected NN. More specifically, we employ one which consists of an input layer with 3 × 224 × 224 neurons, a fully-connected hidden layer with 2×10 3 neurons, and an output layer with 2 neurons [cf. the schematic illustration in Fig. 1 When a sample of our data, i.e., an image of a steady state configuration, is fed to the fully-connected NN, raw pixels of the image are collected by the neurons [cf. circles in Fig. 1(b)] in the input layer. The "fully-connected" structure of this NN is reflected in the fact that all the neurons in the input layer, denoted as the vector x ≡ (x 1 , x 2 , · · · , x i , · · · ) T with x i denoting a single neu- neuron in the output layer shows the "confidence" of the NN in classifying the input sample as "flocking" ("disordered"). (c) Schematic illustration of the convolutional NN. The convolutional kernel (illustrated as a square pyramid) scans across the image I (illustrated as a square sheet) and outputs the extracted features (illustrated as a cube). After several convolutional layers, the "advanced" features from the input sample are finally fed to the classifier, i.e., a fully-connected NN. (d) Schematic illustration of the residual NN, whose key structure is reflected in its "shortcut connections" (illustrated as dashed lines) between different convolutional layers. See text for more details. ron, are "connected" [cf. lines in Fig. 1(b)] to every neuron h i in the hidden layer, and all neurons in the hidden layer, denoted as the vector h ≡ (h 1 , h 2 , · · · , h i , · · · ) T , are "connected" to every neurons y i in the output layer. Mathematically, this "fully-connected" structure is reflected exactly in the expressions for h i and y i , i.e., , where we see that the value of each neuron explicitly depends on the values of all neurons in the previous layer. Here, the rectified linear unit (ReLU) and the sigmoid function serve as the nonlinear "activation functions" [2,3]. w (so-called "weights") and b (so-called "biases") are the "learnable parameters" of the NN to be optimized in the training process [2,3].
For our binary classification with respect to the flocking phase and the disordered phase, the output neurons y = (y 1 , y 2 ) show the "classification confidence" of the NN. For instance, the output neurons y = (0.9, 0.1) show that the NN has 90% (10%) confidence in classifying the input sample as "flocking" ("disordered"). We provide a "suggestion"ỹ = (1, 0) (ỹ = (0, 1)) while labeling a sample as "flocking" ("disordered"). Then, the error of the output y compared to the labelỹ can be quantified by the cost function S = − iỹ i ln y i [2,3], which assumes the form of the "cross entropy". During the training process, all the learnable parameters of the NN are optimized by minimizing the cost function S traversing the whole "training set". After several iterations of training, the fully-connected NN with the optimized learnable parameters is ready to classify samples in the "test set" with respect to the flocking phase and the disordered phase.
Moreover, as a direct consequence of the so-called "universal approximation theorem" in the mathematical theory of NNs [20][21][22], fully-connected NNs with at least a single fully-connected hidden layer can universally approximate any continuous function on compact subsets of R n . Therefore, they are usually employed by output parts of other NN architectures as the "classifiers" that output the final classification confidence [cf. for instance, Figs. 1(c, d)].

Convolutional NN
Another type of NNs employed in this work is the convolutional NN. More specifically, we employ a typical convolutional NN called "AlexNet" [23], which contains an input layer with 3×224×224 neurons, 5 convolutional layers, two fully-connected hidden layers with 9216 and 4096 neurons, and an output layer with 2 neurons [cf. the schematic illustration in Fig. 1(c)].
The key feature of the convolutional NN is that it extracts features from the input sample by the "convolution operation" [2,3]: Let the tensor O (tensor I) be the output (input) of a 2D convolution operation, then O k ′ i,j = k,p,q I k i−p,j−q K k,k ′ p,q , where tensor K is the convolutional kernel with its elements usually referred as "weights". Here, the subscripts represent the 2D coordinates, and the superscripts are indices for the so-called "channels" of the convolutional layer [2,3]. This convolution operation is schematically illustrated in Fig. 1(c) for the special case with one convolutional layer following the input layer, where the convolutional kernel K [cf. the square pyramid in Fig. 1(c)] scans across the image I [cf. the square sheet in Fig. 1(c)] and outputs the extracted features [cf. the cube in Fig. 1(c)]. These extracted features, i.e., all the elements of the tensor O, are further processed with a nonlinear activation function (e.g., ReLU) and used as the final output tensor O of the convolutional layer, i.e., for instance, . Moreover, in certain cases, the final output tensor O of the current convolution layer may undergo additional processing, such as the so-called "pooling", before it is used as the input of the next convolutional layer [2,3]. After several convolutional layers, the "advanced" features from the input sample are finally fed to the classifier, i.e., a fully-connected NN, with the output neurons y = (y 1 , y 2 ) of the classifier showing the classification confidence.
Here, the learnable parameters of the convolutional NN are all the convolutional kernels and all the weights and biases in the fully-connected classifier. They are optimized in a similar way in the training process as the one of training the fully-connected NN.

Residual NN
The last type of NNs employed in this work is a simple version of the standard residual NN [24], which contains an input layer with 3×224×224 neurons, 18 convolutional layers, a fully-connected hidden layer with 512 neurons, and an output layer with 2 neurons [cf. the schematic illustration in Fig. 1

(d)]
In fact, one could regard residual NNs as a special type of convolutional NNs. The key feature of the residual NNs is that they use "shortcut connections" [cf. dashed lines in Fig. 1(d)] between convolutional layers. Mathematically, the shortcut connection corresponds to x ′ = F convolutions (x) + F shortcut (x), where x and x ′ are the features (the intermediate outputs) before and after the shortcut connection, respectively. F convolutions (x) denotes the output after one or more convolutional layers, and F shortcut is a linear projection operation acting on x with the dimension of its output matching the one of F convolutions (x). More specifically, this linear projection is usually performed through the convolution operations employing kernels with a small number of weights to be "learned" in the training process [24]. Therefore, learnable parameters of the residual NN consist of learnable parameters involved in the "shortcut connections", all the convolutional kernels, and all the weights and biases in the fully-connected classifier.
Finally, it is worth noting that using shortcut connections, the residual NN can usually avoid the technical troubles of "vanishing gradients" or "exploding gradients" during the training process that can be caused by the large number of layers [24]. Therefore, this type of NNs can be designed to possess more than a hundred convolutional layers, indicating its remarkable ability in feature extraction that can be utilized in physics research.

III. RESULTS
A. Identification of the flocking transition with the NNs trained via unsupervised learning Let us start with an unsupervised learning approach called "learning by confusion" proposed in Ref. [8] to identify critical points of flocking transition. In the following, we first outline how NNs are trained within this approach in our case, then we discuss the predictions of the flocking transition point from the trained NNs.
To train the NNs, a testing binary classification rule is first imposed in order to label all the samples in the data set. More specifically, we can pick any noise level η ′ c and label all the samples in the three data sets, namely, the "training set", the "validation set", and the "test set", with their corresponding noise levels below (above) η ′ c as "flocking" ("disordered"). After this proposed labeling, all samples in the "training set" and the "validation set" are used in the standard training process (cf. Ref. [2,3] and related discussions in Sec. II B) to optimize the learnable parameters of the NN employed.
After the above training process, the NN's performance or accuracy of its predictions is measured according to the afore labeled "test set". More specifically, the trained NN receives every sample in the "test set" as input, and gives whether the sample is in the flocking phase as output, which is further compared with the label of the sample itself. Therefore, one can obtain the accuracy P (η ′ c ) of the NN's predictions that corresponds to the test binary classification rule associated to η ′ c , whose explicit form reads where M test-set is the total number of samples in the "test set" and M accurate (η ′ c ) is the number of samples that have been classified "correctly" by the NN, i.e., samples whose corresponding outputs of the NN match their afore proposed labels determined by η ′ c . Repeating the above two procedures, i.e., training and prediction accuracy measurement, for different proposed values of η ′ c , one can thus establish the η ′ c dependence of the prediction accuracy of the NN employed. In Fig. 2, P (η ′ c ) is shown for the three different types of NNs, respectively. Except the two trivial choices η ′ c = 0 and η ′ c = 1 which indicate essentially no classification is performed [8,9], for any η ′ c that does not match the physical flocking transition point η c , its corresponding way of labeling the samples inevitably "confuses" the NN in the training process by the wrong labels and hence lower the prediction accuracy of the trained NN [8,9]. Therefore, the accuracy P (η ′ c ) is expected to assume a nontrivial maximum when η ′ c matches the physical flocking transition point η c . As we can see from Fig. 2, for the three types of NNs used in this work, irrespective of their different inner structures, all their P (η ′ c ) curves reach the nontrivial maximum at the same proposed critical noise value with η ′ c = 0.63, indicating the physical flocking transition point η c = 0.63 ± 0.01 which matches well with the prediction from direct numerical simulations [cf. Fig. 1(a)].

B. Identification of the flocking transition with the NNs trained via supervised learning
Now let us switch to a supervised learning approach that has been widely used in identifying critical points of various phase transitions [4,6,7]. In the following, we first outline how NNs are trained in our case, then we discuss the predictions on the flocking transition point from the trained NNs.
To train the NNs, a portion of samples in the "training set" and the "validation set" are first properly labeled according to the prior physical knowledge concerning the system. This is in sharp contrast to the "learning by confusion" approach, where all samples in the data set are labeled according to a series of arbitrarily proposed binary classifications. More specifically, since we know that the system must be in the flocking (disordered) phase at very low (high) noise levels, we can safely label the samples whose corresponding noise levels are below (above) certain threshold η l (η h ) as "flocking" ("disordered"). While for samples with their corresponding noise levels lying between the two thresholds η l and η h , they are left unlabeled and in fact never used in the training process. As shown in Fig. 3, for the case under consideration, we choose η l = 0.2 and η h = 0.8, i.e., for training samples with their corresponding noise levels located within the interval [0, η l ] ([η h , 1]) are labeled as "flocking" ("disordered") as shown by the shaded regions in Fig. 3. After labeling, these properly labeled training data are used in the standard training process (cf. Ref. [2,3] and related discussions in Sec. II B) to optimize the learnable parameters of the NN employed.
After the training, different NNs are employed to predict the flocking transition point. More specifically, the trained NN receives every sample of the "test set" as input, and outputs the "confidence" of whether the input sample is in the flocking (disordered) phase, i.e., the trained NN acts as a two-dimensional vector-valued function (C flocking (I i (η)), C disordered (I i (η))) T . Here, I i (η) denotes the ith image sample in the "test set" with its corresponding noise level being η, and C flocking(disordered) (I i (η)) is the "flocking (disordered) confidence" of the image sample I i (η). Therefore, one can establishes the noise level dependance of the "flocking (disordered) confidence" for the complete "test set", i.e., with N (η) being the total number of samples in the "test set" with their corresponding noise level being η. In Fig. 3, the noise level dependance of the "flocking (disordered) confidence" is shown for the three different types of NNs, respectively. Since at the flocking transition point, the system can be either in the flocking phase or the disordered phase, the trained NN naturally gives equal confidence of flocking and disordered. Therefore, the critical noise level η c of the flocking transition can be identified by the noise level with equal flocking and disordered confidence, which corresponds, for instance, the intersection point of the two curves in each plot of Fig. 3.
As we can see from Fig. 3, the predicted value for η c from different types of NNs trained via the supervised learning approach matches very well with the prediction from direct simulations [cf. Fig. 1(a)] and hence also agrees very well with each other. The relative difference among the predictions of η c by the three types of NNs is less than 5 (the fully connected NN predicts η c = 0.630, the convolutional NN predicts η c = 0.632, and the residual NN predicts η c = 0.632). Noticing that these three types of NNs in fact assume quite different inner structures [cf. Figs. 1(b, c, d)], this good mutual agreement clearly manifests that NNs trained via supervised learning is insensitive to the inner structure of NNs, hence robust in identifying the first-order phase transition point. As a final remark, it is worth noting that, compared to the "learning by confusion" approach, identifying the phase transition point within this supervised learning approach consumes much fewer computation resources. For instance, the computation time cost in ob-taining a data point shown in Fig. 2 is similar to the one cost in establishing a complete plot shown in Fig. 3. This mainly attributes to the fact that the training process of the supervised learning approach explicitly exploits the prior physical knowledge of the system, i.e., training samples with their corresponding noise levels being below (above) η l (η h ) are properly labeled according to their physical properties.

IV. CONCLUSION AND OUTLOOK
We systematically apply three different representative types of NNs, ranging from the simple fully-connected NN, over the convolutional NN, to the more advanced residual NN, trained via either the unsupervised learning approach or the supervised learning one, to the problem of identifying the first-order flocking transition of the self-propelled particles under influences from environmental fluctuations. We find that after training, any of these three types of NNs, without "special fine tuning" on its inner NN architecture, is able to extract the critical extrinsic noise level of the flocking transition. We further find these predicated critical noise levels by different NNs trained via different approaches agree well with each other, and with the estimation from direct numerical simulations. This suggests that NN based machine learning is a promising generic tool to investigate rich physics in scenarios associated to first-order phase transitions and NEQ systems. We believe our work will stimulate further efforts in exploring machine techniques to investigate rich physics in these scenarios. For instance, it is intriguing to apply machine learning techniques to revel the characteristic NEQ critical scaling behavior in NEQ many-body systems, such as the prototypical Kardar-Parisi-Zhang NEQ scaling behavior of the growing interface [26]. Moreover, noticing nontrivial scaling could arise for hysteresis associated to first-order phase transitions [27], it is also intriguing to apply machine learning techniques to investigate nontrivial scaling behavior of hysteresis phenomena.