{'fraction_non_alphanumeric': 0.03808263490663138, 'fraction_numerical': 0.01754643924913003, 'mean_word_length': 4.745987045902563, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
An advanced hybrid deep adversarial autoencoder for parameterized nonlinear fluid flow modelling

Considering the high computation cost produced in conventional computation fluid dynamic simulations, machine learning methods have been introduced to flow dynamic simulations in recent years. However, most of studies focus mainly on existing fluid fields learning, the prediction of spatio-temporal nonlinear fluid flows in varying parameterized space has been neglected. In this work, we propose a hybrid deep adversarial autoencoder (DAA) to integrate generative adversarial network (GAN) and variational autoencoder (VAE) for predicting parameterized nonlinear fluid flows in spatial and temporal space. High-dimensional inputs are compressed into the low-representation representations by nonlinear functions in a convolutional encoder. In this way, the predictive fluid flows reconstructed in a convolutional decoder contain the dynamic flow physics of high nonlinearity and chaotic nature. In addition, the low-representation representations are applied into the adversarial network for model training and parameter optimization, which enables a fast computation process. The capability of the hybrid DAA is demonstrated by varying inputs on a water collapse example. Numerical results show that this hybrid DAA has successfully captured the spatio-temporal flow features with CPU speed-up of three orders of magnitude. Promising results suggests that the hybrid DAA can play a critical role in efficiently and accurately predicting complex flows in future.


Introduction
Numerical simulations of nonlinear fluid flows are used to describe the complex evolution of many physical processes. Accurate simulations of nonlinear fluid flows are of great significance to many fields such as flood and ocean modelling, which systems often exhibit rich flow dynamics in both space and time (Erichson et al., 2019;Hu et al., 2019). The numerical simulations have benefited from the availability of high-resolution spatio-temporal data with recent advances in measurement techniques (Taira et al., 2019), which makes the studies of more complex flows become possible. However, the computational cost involved in solving complex problems is intensive which still precludes the development in these areas. In order to address the issue of high computational cost, this paper proposes a hybrid deep adversarial autoencoder to solve fluid problems in an efficient manner.
Recent advances in machine learning technologies are increasingly of interest for efficiently simulating flows in dynamical systems (Lusch et al., 2018;Lee and Carlberg, 2018;Murata et al., 2019). Machine learning has demonstrated its potential capability in fluid flow applications , such as fluid flow modelling (Geneva and Zabaras, 2019;Ling et al., 2016), flow simulation (Kutz, 2017;Liu et al., 2017), and fluid field reconstruction (Farimani et al., 2017;Kim et al., 2019). For example, Humphrey et al. (2016) utilized a Bayesian artificial neural network (ANN) with a conceptual model to forecast monthly streamflow. Mohan and Gaitonde (2018) developed the Long Short Term Memory (LSTM), a type of recurrent neural network (RNN), to simulate the temporal dynamics of turbulent flows. Kim et al. (2019) adopted a convolutional neural network (CNN) to reconstruct fluid fields. Despite these researches demonstrated to successfully reconstruct flow dynamics, it is noted that they commonly do not take into account the temporal and spatial evolution of inputs or parameters, which is crucial for realistic dynamical systems (Reichstein et al., 2019).
Most recently, the generative adversarial networks (GANs) have been developed for predicting the parameterised nonlinear fluid flows (Farimani et al., 2017). GAN introduced by Goodfellow et al. (2014), recently has emerged as a leading role for recreating the distributions of complex data (Xie et al., 2018). The key feature of GAN is the adversarial strategy in two modules. GAN defines a learning generative network by transforming a latent variable into a state variable using nonlinear functions. Then GAN drives the learning process by discriminating the observed data from the generated data in a discriminator network. Because of the special adversarial architecture, GAN has demonstrated great capability in producing high-resolution samples, mostly in images, e.g., image synthesis (Reed et al., 2016), semantic image editing (Li and Wand, 2016), style transfer (Isola et al., 2017) etc.
For efficient GAN training and accurate spatio-temporal fluid flow prediction, Variaitonal Autoencoder (VAE) (Kingma and Welling, 2013) has been introduced to GAN in this study. VAE has been widely used in various research areas, such as text Generation (Semeniuta et al., 2017), facial attribute prediction (Hou et al., 2017), image generation (Walker et al., 2016;Pu et al., 2016), graph generation (Simonovsky and Komodakis, 2018), music synthesis (Roberts et al., 2017), and speech emotion classification (Latif et al., 2017), etc. The hybrid deep adversarial autoencoder (DAA) developed here takes advantages of both GAN and VAE. GAN allows for training on large datasets and is fast to yield visually and high-resolution images, but the flexible architecture is easy to come with the model collapse problem and generate unreal results (Rosca et al., 2017). The VAE is attractive for achieving better log-likelihoods than GAN (Wu et al., 2016;Mescheder et al., 2017), therefore it encourages the hybrid DAA to better represent all the training data and discouraging mode-collapse problem in GAN (Rosca et al., 2017).
The hybrid DAA developed here is a robust and efficient numerical tool for accurate prediction of parameterised nonlinear fluid flows. The advantages of the hybrid DAA include: • The proposed method exploits spatial features by use of convolutional neural networks. It will be advantageous over the traditional reduced order models (ROMs) (Fang et al., 2017;Xiao et al., 2019) since the high-dimensional datasets are compressed into the low-dimensional representations by nonlinearity functions in a convolutional encoder. In this way, the predictive fluid flows containing high nonlinearity and chaotic nature can be represented by a convolutional decoder.
• The low-dimensional representations, several orders of magnitude smaller than the dimensional size of the original datasets, are applied into the adversarial network for representation learning and parameter optimization, thus accelerating the computation process.
• With the trained hybrid DAA, for any given different inputs, the spatio-temporal features can be automatically extracted in the encoder. Consequently, accurate predictive nonlinear fluid fields can be further obtained in the decoder with an efficient manner. This is the first time that the hybrid DAA is adopted to address parameterised nonlinear fluid flow problems. It will make a breakthrough in predicting accurate nonlinear fluid flows with the high-speed computation.
The reminder of this paper is organised as follows. Methodologies of VAE and GAN are briefly introduced in section 2 and section 3, respectively. The hybrid DAA for parameterised nonlinear fluid fields is detailed described in section 4. section 5 demonstrates the performance of the hybrid DAA using water collapse as a test case. Finally in section 6, conclusions are presented.

Variational Autoencoder
Variational Autoencoder (VAE) is introduced by Kingma and Welling (2013), which combines Bayesian inference with deep learning. The VAE is a generative model which aims to produce the desired local variableh from the underlying latent variable ζ. Mathematically, let p ζ (ζ) be a prior distribution of ζ, and the probability of the local state variablē h be modelled byh where p θ (h|ζ) is the conditional distribution of the local state variableh given ζ, which is modelled by deep neural networks (called decoder) with parameters θ.
In contrast to standard autoencoders, the key property of VAE is the ability to control the distribution of the latent state vector ζ, which is usually modelled by a standard Gaussian distribution N (ζ|0; I) (Kingma and Welling, 2013). In VAE, the probability of the latent vector ζ can be expressed as where q φ (ζ|h) is the conditional distribution of the local state variableh given ζ, which is modelled by deep neural networks (called encoder) with parameters φ.
To achieve the sample reconstruction, the reconstruction loss L rec as the negative expected log-likelihood of the samples needs to be maximized, as following: The difference (called KullbackLeibler divergence) between the distribution of q(ζ|h) and a prior distribution (for example, Gaussian distribution) p ζ (ζ) = N(ζ|0; I) can be quan- The total loss is consisted of the reconstruction loss and the KullbackLeibler (KL) divergence (L vae = L rec + L KL ), which can be minimized by gradient descent algorithms (Kingma and Welling, 2013). Since the KL divergence is non-negative, the total loss can be expressed as: Correspondingly, the full objective function of VAE is as follows where p data (h) is the prior distribution ofh.

Generative adversarial network
The GAN is generally implemented with a minimax game in a system of two players (Goodfellow, 2016). One player is responsible for generating the new samplesh from a random dataset µ , while another player aims to discriminate the real samplesh d from the generated samplesh. The former player is called the generator G and the latter is the discriminator D.
In the discriminator, D(h d ) = 1 if the real samplesh d are accepted while D(G(µ)) = 0 if the generated samplesh (h = G(µ)) rejected. Unlike the autoencoder, GAN is a twoplayer game rather than optimizing one loss function L vae (as in Eq. (6)) in VAE. During the training process of GAN, the parameters in the discriminator are updated by maximizing L D as: while the parameters in the generator are updated by minimizing L G as: where p µ (µ) is a prior distribution for the random dataset µ, and p data (h d ) is the corresponding probability data distribution for the real datasetsh d .
Concretely, the objective function for GAN is shown in Eq. (9): In practice, this optimization of L(G, D) is performed alternately using gradient-based methods, e.g. the adaptive Moment Estimation (Adam) optimizer (Kingma and Ba, 2014).
Given enough capacity, the game converges to a global optimum where D(h d ) = 1 2 everywhere (details shown in Goodfellow et al. (2014)).

Hybrid deep adversarial autoencoder for nonlinear fluid flow modelling
In this paper, for nonlinear fluid flow modeling, a hybrid deep learning fluid model based on deep adversarial autoencoder (DAA) is proposed which is established by com-bining a VAE and a GAN. As shown in Fig. 1(a), the encoder is acted as a generator in the hybrid DAA. In comparison to Eq.(2) in VAE, the generator here defines an aggregated posterior distribution of q(ζ) on the latent state as followings: where q data (h) is the data distribution for the datasetsh. The autoencoder in the hybrid DAA is regularized by matching the aggregated posterior q(ζ) to the prior p(ζ) in Eq.(1).
As described in Fig. 1(a), in the forward propagation process, the inputs µ and the targeted outputsh b are fed into the encoder which generates the corresponding latent states ζ and ζ b respectively. The encoder tries to produce the aggregated posterior distribution q(ζ b ) matched with the prior distribution p(ζ), which can fool the discriminator in GAN.
The latent states ζ b and ζ are then transformed into the reconstructed targeted outputsh b and generated outputsh respectively in the decoder. In the backward propagation process, the parameters in modules: encoder, decoder and discriminator are updated by minimizing the reconstruction loss and maximizing the adversarial loss.
In general, a parameterized partial differential equation for a spatio-temporal fluid flow problem can be written as where M denotes a nonlinear partial differential operator,h(x, µ, t) is the state variable vector (for example, pressure, density, velocity, etc), x represents the spatial coordinate system, µ denotes the parameter vector (for example, model input and boundary condition), t is the time and F is the source term.
In spatio-temporal fluid flow simulations, the state variable vectorh represents the fluid flow distribution during a specified simulation period [0, t Nt ] as whereh tn t is the state variable vector at time level t nt (n t ∈ [0, N t ], t nt ∈ [0, t Nt ], N t is the number of timesteps).
In a spatio-temporal discretisation form, the state variable vectorh in Eq.(12) can be rewritten as:h where N x denotes the number of points in scalar grids of the computational domain Ω.
The parameter vector µ in Eq.(11) at a spatial space Ω can be expressed: Given the parameter vector µ = [µ 1 , µ 2 , ..., µ Nx ] as inputs and the state variable vector as the targeted outputs, the model architectur of the hybrid DAA are shown in Fig. 1.

Numerical examples
The example used for validation of the hybrid DAA is a case of water column collapse.
It is a benchmark test case for multi-material models, which is also known as a dam break problem. The dam-break flow problem has been of great importance in hydraulic engineering and hydropower generation. Dam-break flows are complex and generally discontinuous with abrupt variations of flow velocity and water depth (Seyedashraf et al., 2017). The flooding induced by the dam-break flows causes great loss of human life and property as well as damaging the ecosystem in the downstream area.
In this example, the dam-break experiment conducts in a tank with the length 3.22m, the height 2m, and the depth 1m (Zhou et al., 1999). The simulated reservoir of water is held behind a barrier at one end of the tank. For no variations are introduced in the third dimension, the experiment is reproduced in the horizontal and vertical dimensions within the domain area Ω. Fig. 2 shows the mesh in the domain area Ω. The dam break problem is simulated using the unstructured mesh finite element fluid model (F luidity) (Pain et al., 2001)(referred as the original high fidelity model). The densities of water and air are 1, 000kgm −2 and 1kgm −2 respectively. The scalar fields a k representing the volume fraction is introduced to distinguish the two materials. As shown in Fig. 2, the interface between the water (the yellow area, a k = 1) and air (the blue area, a k = 0) is delineated by contours at a k of 0.025, 0.5 and 0.975. In this case, the simulation period is [0, 1.85]s, with a timestep = 0.025s. Thus, the targeted fluid flowsh d were obtained by running the original high fidelity model, with a unstructured mesh of 19097 nodes.

Model appplication
Data collection: In this case, for training the hybrid DAA, the length L and height H of the tank are selected as the input parameters (as shown in Fig. 3). The length of the tank L is ranged from 1.2 m to 1.6 m, and the height H of the tank varies from 0.6 m to 1.0 m.
A series of sizes as inputs µ and the corresponding solution snapshotsh d (a total of 3375 snapshots of 45 pairs of input-output) were obtained by running the high fidelity model, which can be re-written in a time discretized form: 0,44], N x = 19097, N t = 75). (16) The training and validated input-output pairs ϑ tr and φ tr (where µ a ∈ R Nx ,h d,a ∈ R Nx×N t , a ∈ [0, 38], N x = 75, N t = 19097) are selected for model training and parameter optimization, while the remained inputs (ϑ\ϑ tr ) (where µ a ∈ R Nx , a ∈ [0, 6], N x = 19097) are used for model prediction.   Prediction of temporal nonliear fluid flows: Fig. 6 shows the temporal variation of pressure solution predicted by the two models. Before the water collapse happens, the points P1, P2, P5 and P6 are located in the water area (as marked with a blue rectangle in Fig. 3), while other four points P3, P4, P7 and P8 are placed in the area filled with air. Model performance has been evaluated by comparison of pressure variation in these detector points. In the Fig. 6(a), (b), (e) and (f), the pressure values at four points are slightly increased and then gradually decreased as the water collapse happens. As for detector P3, it is beyond the scale of water front motions before t = 1.5s, and the pressure trend in Fig. 6(c) is not so obvious. After t = 1.5s, influenced by the overturning water, the pressure at the detector P3 starts to fluctuate. Compared to detector P3, it can be noted that the pressure values at detector P4, P7 and P8 are uprising when the water drops down the horizon into rightward direction. The sequent water experiences dropping, traveling, rising up along the right vertical wall, and dropping again processes. It can be observed that the curves of pressure from the hybrid DAA achieve a good agreement with that of the original high fidelity model, except for the under-predictions of dropping again process after t = 1.5s. However, it is also a challenge problem for numerical simulation when the water hits at the right wall and again meets the horizontal free surface (Park et al., 2009). Model performance anaylsis: To further evaluate the performance of the hybrid DAA, the correlation coefficient and RMSE are calculated in temporal and spatial space, respectively. The error analysis of pressure solutions in temporal space is shown in Fig. 7. It is evident that the values of RMSE are between 0pa to 2000pa while the correlation coefficients are above 90%. In light of fluid field prediction in temporal space, these results demonstrate that the hybrid DAA performs well and the predicted fluid fields are in good agreement with the true fluid fields. Fig. 7. The correlation coefficients and RMSE of pressure between the generated fields and original high fidelity fields during whole simulational period. Fig. 8 illustrates the model performance in spatial space. It can be noted that the RMSE is not beyond 1900pa in the whole domain area Ω, which demonstrates the capability of hybrid DAA to predict accurate fluid flows. However, the correlation coefficient is negative in some area, where is affected by the water jet. When the water front strike the right vertical wall, the water jet causes a sudden rise of pressure and air entrainment (Park et al., 2009). The phenomena is decribed in Fig. 6(c), where depicts the pressure fluctuation after t = 1.5s. Except for the area influenced by the interface of multi-materials, the correlation coefficient is beyond 0.9 in the domain area Ω.

Model efficiency
The simulations of the hybrid DAA and the original high fidelity model were preformed on Intel Xeon(R) CPU@ 3.60GHz with a 449.5 GB memory. The computation time required for the hybrid DAA in online prediction process is 21.76s, while 8836.67s for running the original high fidelity model. It can be seen that the CPU time for running the hybrid DAA is reduced drastically by three orders of magnitude in comparison to the original high fidelity model.

Conclusions
In this work, a hybrid DAA method has been, for the first time, used to predict nonlinear fluid flows in varying parameterized space. For any given input parameters µ, the hybrid DAA is capable of predicting accurate dynamic nonlinear fluid flows and remains high efficiency in simulation, which takes both advantages of VAE and GAN.
The performance of the hybrid DAA has been demonstrated by a water collapse test case. To evaluate the model accuracy, a detailed comparison between the original high fidelity model (Fluidity) and the hybrid DAA has been undertaken. The accuracy assessment has also been performed through the correlation coefficient and RMSE. The numerical simulations show that the hybrid DAA exhibits good agreement with the original high fidelity model, in both space and time. Additionally, a significant CPU speed-up has been achieved by the hybrid DAA.
The hybrid DAA is an efficient and robust tool for parameterized modelling and prediction of nonlinear fluid flows. It provides a wide range of applications, for instance, risk response management in emergencies and natural hazards (e.g. flooding, dam break, etc), real-time decision making. Future work will be focused on the predictive ability of leadtime by the hybrid DAA and more complex fluid problems can also be examined with this model.