{'fraction_non_alphanumeric': 0.03595890410958904, 'fraction_numerical': 0.00926214196762142, 'mean_word_length': 4.951366373320982, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Physical Benchmarking for AI-Generated Cosmic Web

The potential of deep learning based image-to-image translations has recently drawn a lot of attention; one intriguing possibility is that of generating cosmological predictions with a drastic reduction in computational cost. Such an effort requires optimization of neural networks with loss functions beyond low-order statistics like pixel-wise mean square error, and validation of results beyond simple visual comparisons and summary statistics. In order to study learning-based cosmological mappings, we choose a tractable analytical prescription - the Zel'dovich approximation - modeled using U-Net, a convolutional image translation framework. A comprehensive list of metrics is proposed, including higher-order correlation functions, conservation laws, topological indicators, dynamical robustness, and statistical independence of density fields. We find that the U-Net approach does well with some metrics but has difficulties with others. In addition to validating AI approaches using rigorous physical benchmarks, this study motivates advancements in domain-specific optimization schemes for scientific machine learning.


Introduction
In the current era of 'precision cosmology', cosmological N-body simulations play a key role in the study of the large-scale structure of the universe, and are essential to interpreting the unprecedented amount of observational data available from sky surveys [1]. In exploring cosmological parameter space, it is computationally challenging to evolve billions and trillions of particles at high enough resolution, even for a single simulation. Since large numbers of simulations are needed for tasks such as covariance estimation [2], the computing requirements can easily become prohibitive.
Deep learning (DL) has exhibited great potential in learning highly complex functions and mappings, thereby attracting attention in various scientific domains, including cosmology [3]. In particular, Figure 1: Comparison of projected densities between the predicted cosmic web and ground truth. Densities are derived from the displacement field with a Cloud-In-Cell (CIC) method, and summed over one axis. The 'input' panel (first panel) presents the density field projection of the early (z = z 0 ) snapshot. The colorbar in each panel shows the magnitude of the dark matter density ρ(x).
if DL methods can successfully capture the complexity of cosmic evolution, they can directly address the computational bottleneck, and even eliminate it in some cases. However, issues such as interpretability, as well as the ability of DL models to make sufficiently accurate and physically meaningful predictions, are yet to be fully explored. For example, can DL models, without any domain knowledge of cosmology, capture all the intricacies of nonlinear gravitational clustering? And in what way can AI-based emulators complement standard numerical approaches?
In this ongoing work, we model cosmic evolution using the widely adopted U-Net approach [4], applying it to a theoretically well-understood prescription, the Zel'dolvich approximation (ZA) [5]. Since the ZA is a simple dynamical mapping scheme, it has the benefit of providing a clear physical picture while enabling a comprehensive study of the neural network's physical interpretability.
2 Approximation for structure formation of the Universe Dynamical approximation : The Zel'dolvich approximation is the first-order solution to the Lagrangian perturbation theory of structure formation. It describes the motion of mass elements: an initially uniform distribution given by Lagrangian coordinates q is displaced by where x(r) are the comoving coordinates, b(t) is the linear growth rate of fluctuations and S( q) = ∇Φ( q) is the gradient of the initial gravitational potential Φ( q). The potential Φ( q) is determined by the primordial density fluctuations δ(x) via the Poisson equation, ∇ 2 Φ = 4πGδ, where G is the Newtonian constant of gravitation. Initial density perturbations are realizations of a Gaussian random field, which is fully specified by a (given) power spectrum. Uniformly placed particles representing mass elements are then moved according to the ZA. Generation of the Dataset: For an initial power spectrum P (k), the displacement field S(q) is generated using FFT-based techniques, and the particles are transported from a regular grid via ZA displacements. We generate 2000 pairs of ZA-evolved particles at two different time steps of the evolution, with respective scale factors a 0 = 0.0464 (an "early" snapshot with redshift z 0 = 20.5) and a 1 = 0.215 (a "late" snapshot with redshift z 1 = 3.6). The evolution of the so-called "cosmic web" in between these two moments is given by the ZA (from Equation 1), solely determined by the initial potential gradient. The displacement fields at these two times form the basis of the training/validation datasets. Data is split such that 1000 pairs are used for training, 500 for validation and 500 for testing. Each dataset is generated with a different random seed to ensure their statistical independence. The dataset consists of 64 3 N-body particles in a volume of 64 h −1 Mpc and their displacements in three spatial dimensions.

Snapshot-to-snapshot translation using convolutional neural networks
A number of DL-based unsupervised generative models and supervised interpolation models have been applied in cosmological data creation [6,7,8]. Except for a few applications where the loss functions are tailored to the physical problem, most of the applications are domain-agnostic. That is, all the information about the underlying physics is entirely learned from the training data. These data-driven models have shown good accuracy with validation datasets, albeit with respect to metrics that resemble the loss function. Architecture: We choose an image-to-image translation framework [9] originally used for biomedical images and recently applied in a cosmological setting [10,4,11,12,13]. We adopt the 3D U-Net architecture used in [4] with 15 convolution and deconvolution layers. The architecture has a contracting path and then an expanding path. A basic block consists of a convolution layer, batch normalization layer and a Rectified Linear Unit (ReLU) layer. Mean squared error between the Lagrangian co-ordinates is used as the loss function, with L2 regularization. We refer the reader to Refs. [9,4] for further details of the network architecture.
Model Training: During the training of the 3D U-Net, a pair of initial-final displacement fields is selected randomly from the dataset as the input, and the neural network is trained to learn the mapping between the z 0 and z 1 displacement fields. After the training is completed, the model is run on a set of test data, independent from training or validation, in order to compare model predictions with the ground truth (the ZA result).

Validation Metrics
Once the loss has converged (after 10 training epochs), we first visualize the reconstructed cosmic web. Figure 1 shows the U-Net generated density (projected along the x-axis) of the cosmic density field, demonstrating a reasonably good agreement with the ZA result. We next implement a number of more quantitative tests. In particular, we look at measures that quantify the matter distribution in the universe and topological connectivity of large scale structures. Details of these metrics are given below and summarized in Fig. 2.
Pixel-wise Comparison: The most straightforward comparison is the relative error in predicted densities or displacements. For a density field ρ(x), we define a relative error |ρ(x) pred − ρ(x) truth |/ρ(x) truth , where ρ(x) pred is the U-Net prediction, and ρ(x) truth is the ZA result. The last panel of Fig. 1 shows the relative difference for one of the realizations. In addition, the one-point probability distribution function (PDF) is used to assess the prediction field, which is also an important measure for nonlinear clustering in cosmic structure. The density PDF measurements of prediction and ground truth are shown in the first panel of Fig. 2. While relatively unbiased, we note that the density PDF error rises with density, which is concerning if the simulation is expected to capture correct clustering physics.
Matter Power Spectrum: The power spectrum is the Fourier transform of the 2-point correlation function in real space, and is widely used as a statistical measure of the density field. The matter overdensity δ(x) = (ρ(x) − ρ)/ρ, where ρ is the mean density. Writing its Fourier transform dual as δ(k), the power spectrum P (k) is defined by δ(k)δ(k ) = (2π) 3 P (k)δ 3 (k − k ). Measurements of the predicted power spectra and ground truth P (k) are shown in the second panel of Fig. 2, where we observe differences of less than 5%, although there is a k-dependent bias in the error behavior.
Three-Point Correlation -Bispectrum: Apart from two-point correlation functions, higher-order statistics can also be used to characterise the density field, and are especially useful for studying the late stages of structure formation where the evolution is highly nonlinear and non-Gaussian. Adapting similar symbol conventions as above, the bispectrum B(k 1 , k 2 , k 3 ) [14] is defined as: δ(k 1 )δ(k 2 )δ(k 3 ) = (2π) 3 B(k 1 , k 2 , k 3 )δ 3 (k 1 + k 2 + k 3 ). Equilateral triangles in k space are selected for our bispectra analysis shown in the third panel of Fig. 2. Deviation of the bispectra of the U-Net generated fields with respect to ZA bispectra is considerably higher compared to the lower order metrics of the one-point PDF and matter power spectra.
Topological metrics -Percolation Analysis: Topological statistics can be used to directly address the issue of connectivity of spatial structures, e.g., clusters of galaxies [15]. For a given density threshold, the volume fraction of the excursion set f ES (i.e., the regions over the threshold) and the volume fraction of the largest region f 1 are computed. The behavior of f 1 /f ES as a function of f ES (last panel of Fig. 2) displays a rapid transition from zero to 1; this signifies a topological shift from isolated clusters to a fully connected field. For the input density field (z = z 0 ), the percolation transition happens at a different density threshold than z = z 1 case, indicating a change in the topological behaviour. For this metric the NN results are very good and in excellent agreement with those from the ZA.
Cross-power Test: The accurate cosmic evolution follows fixed dynamical rules, independent of the initial conditions. However, the neural network emulating the evolution might show discernible bias inherited from the training dataset. Specifically, we need to investigate whether the NN prediction induces otherwise non-existent correlations among the outputs, in contrast to the ZA-evolution of fields, where independent initial conditions should result in independent evolved fields. An example test of a cross-power spectral measurement is shown in Fig. 3(b), where we generate two initial conditions, independent of the training set, measure their cross-power spectra, then evolve them separately by ZA and NN, and then measure the output cross-power between them again (Fig. 3(a) illustrates this validation route). From Fig. 3(b), the cross-power shows that, at least for 2-point statistics, there are no hidden correlations among the NN-predicted density fields.
While DL-based translation networks and generative models have demonstrated significant potential, critical tests have to be made before they are applied in high precision scientific fields like cosmology (where we are interested in error metrics aiming at accuracies of better than 1% [16]). In this work, we have demonstrated a subset of the validation schemes to motivate the scientific machine learning community towards using comprehensive sets of benchmarking tools. Starting from a tractable approximation to the complex dynamics of cosmic structure formation, we demonstrate the use of metrics that were not part of the optimization of the neural network. For instance, topological transitions and bispectra are higher-order emergent features, agreement with which is not trivial in NN-based structure formation models. These checks not only serve as robust validation tools, but also provide a clear direction for designing future cosmic structure emulation schemes.
We note that although the DL results appear to be promising and close to the true physical results, in order for neural networks to compete with simulations, eventually all errors must be less than 1% for clustering metrics, and the spatial dynamic range of the 3D NN results must be significantly extended, from a part in a hundred to parts per million. An interesting and long road lies ahead.