{'fraction_non_alphanumeric': 0.03296513634794615, 'fraction_numerical': 0.012747176882489275, 'mean_word_length': 4.904644052991702, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Deep-learning inversion: a next generation seismic velocity-model building method

Seismic velocity is one of the most important parameters used in seismic exploration. Accurate velocity models are key prerequisites for reverse-time migration and other high-resolution seismic imaging techniques. Such velocity information has traditionally been derived by tomography or full-waveform inversion (FWI), which are time consuming and computationally expensive, and they rely heavily on human interaction and quality control. We investigate a novel method based on the supervised deep fully convolutional neural network (FCN) for velocity-model building (VMB) directly from raw seismograms. Unlike the conventional inversion method based on physical models, the supervised deep-learning methods are based on big-data training rather than prior-knowledge assumptions. During the training stage, the network establishes a nonlinear projection from the multi-shot seismic data to the corresponding velocity models. During the prediction stage, the trained network can be used to estimate the velocity models from the new input seismic data. One key characteristic of the deep-learning method is that it can automatically extract multi-layer useful features without the need for human-curated activities and initial velocity setup. The data-driven method usually requires more time during the training stage, and actual predictions take less time, with only seconds needed. Therefore, the computational time of geophysical inversions, including real-time inversions, can be dramatically reduced once a good generalized network is built. By using numerical experiments on synthetic models, the promising performances of our proposed method are shown in comparison with conventional FWI even when the input data are in more realistic scenarios. Discussions on the deep-learning methods, training dataset, lack of low frequencies, and advantages and disadvantages of the new method are also provided.


INTRODUCTION
Currently, velocity-model building (VMB) is an essential step in seismic exploration because it is used during the entire course of seismic exploration including seismic data acquisition, processing, and interpretation. Accurate subsurface-image reconstruction from surface seismic wavefields requires precise knowledge of the local propagation velocities between the recording location and the image location at depth. Good velocity models are prerequisites for reverse-time migration (Baysal et al., 1983) and other seismic imaging techniques (Biondi, 2006). Estimated velocity models can also be used as initial models to recursively generate high-resolution velocity models with optimization algorithms (Tarantola, 2005). Many of the explored techniques such as migration velocity analysis (Al-Yahya and Kamal, 1989), tomography (Chiao and Kuo, 2001), and full-waveform inversion (FWI) (Tarantola, 1984;Mora, 1987;Virieux and Operto, 2009) share the same purpose of building more accurate velocity models.
Traditional tomography methods (Woodward et al., 2008), including reflection tomography, tuning-ray tomography, and diving-wave tomography (Stefani, 1995), are widely used for migration of seismic reflection data in building three-dimensional (3D) subsurface velocity models. Such methods have worked sufficiently well in most cases. Seismic inversion is performed by means of wave inversion of a simple prior model of the subsurface, and by using a back propagation loop to infer the subsurface geological structures (Tarantola, 1984).
Typically, FWI is a data-fitting procedure used in reconstructing high-resolution velocity models of the subsurface, as well as other parameters that govern wave propagation, from the full information contained in seismic data (Virieux and Operto, 2009;Operto et al., 2013). In FWI, active seismic sources are used to generate seismic waves, and geophones are placed on the surface to record the measurements. An inverse problem is formulated to combine the measurements with the governing physics equations to obtain the model parameters. Numerical optimization techniques are utilized to solve for the velocity models.
When a suitably accurate initial model is provided, FWI is highly effective for obtaining a velocity structure through iterative updates. Efforts have been made recently to overcome the limitations in FWI. Even though these conventional methods have shown great success in many applications, they can be limited in some situations owing to a lack of low-frequency components as well as computational inefficiency, subjective human factors, and other issues. Additionally, iterative refinement is expensive when used in the entire workflow. Thus, a robust, efficient, and accurate velocity-estimation method is needed to address these problems.
Machine learning (ML) is a field of artificial intelligence that uses statistical techniques to give computer systems the ability to "learn" from big data. ML has shown its strength in many fields including image recognition, recommendation systems (Bobadilla et al., 2013), spam filters (Androutsopoulos et al., 2000), fraud alerts (Ravisankar et al., 2011), and other applications. Furthermore, ML has a long history of applications in geophysics. Nonlinear intelligent inverse technologies have been applied since the mid-1980s. Röth and Tarantola (1994) first presented an application of neural networks to invert from the time-domain seismic data to a depth profile of acoustic velocities. They used pairs of synthetic shot gathers (i.e., a set of seismograms obtained from a single source) and corresponding one-dimensional (1D) velocity models to train a multi-layer feed-forward neural network with the goal of predicting the velocities from new recorded data. They showed that the trained network can produce high-resolution approximations to the solutions of the inverse problem. In addition, their method can invert the geophysical parameters in the presence of white noise. Nath et al. (1999) used neural networks for cross-well traveltime tomography. After training the network with synthetic data, the velocities can be topographically estimated by the trained network with the new cross-well data. In recent years, most ML-based methods have focused mainly on pattern recognition in seismic attributes (Zeng, 2004;Zhao et al., 2015) and facies classifications in well logs (Hall and Brendon, 2016). In the work of Guillen et al. (2015), the authors proposed a novel workflow to detect salt bodies based on seismic attributes in a supervised learning method. An ML algorithm (i.e., Extremely Random Trees Ensemble) was used to train the mapping for automatically identifying salt regions. They concluded that ML is a promising mechanism for classifying salt bodies when the selected training dataset has a sufficient capacity for describing the complex decision boundaries. Jia and Ma (2017) used ML with supported vector regression (Cortes and Vapnik, 1995) for seismic data interpolation. Unlike the conventional methods, no assumptions are imposed on ML-based interpolation problems. On the basis of the above work, Jia et al. (2018) proposed a method based on the Monte Carlo method (Yu et al., 2016) for intelligent reduction of training sets. In that study, representative patches of seismic data were selected to train the method for efficient reconstructions.
Deep learning (DL) (LeCun et al., 2015;Goodfellow et al., 2016), a new branch of machine learning, has drawn widespread interest by showing outstanding performance for recognition and classification (Greenspan et al., 2016) in image and speech processing. Recently, Zhang et al. (2014) proposed to use of a kernel regularized least-squares method (Evgeniou et al., 2000) for fault detection from seismic records. The authors used toy velocity models to generate seismic records and set the records and the velocity models as inputs and labels in the training set. The numerical experiments showed that this method obtained meaningful results. Wang et al. (2018a) developed a salt-detection technique from raw multi-shot gathers by utilizing a fully convolutional neural network (FCN). The testing performance showed that salt detections is much faster and efficient by this method than traditional migration and interpretation. Lewis and Vigh (2017) investigated a combination of DL and FWI to improve the performance for salt inversion. In that study, the network was trained to generate useful prior models for FWI by learning features relevant to earth model building from a seismic image. The authors tested this methodology by generating a probability map of salt bodies in the migrated image and incorporating it in the FWI objective function. The test results showed that this method is promising in enabling automated salt body reconstruction using FWI. In the work of Araya-Polo et al.
(2017), a deep neural network (DNN)-based statistical model was used to automatically predict faults directly from synthetic two-dimensional (2D) seismic data. Inspired by this concept, Araya-Polo et al. (2018) proposed an approach for VMB. One key element of this DL tomography is the use of a feature based on semblance that predigests the velocity information. Extracted features are obtained before the training process and are used as DNN inputs to train the network. Mosser et al. (2018) used a generative adversarial network (Goodfellow et al., 2014) with cycle-constraints (Zhu et al., 2017) to perform seismic inversion by formulating this problem as a domain-transfer problem. The mapping between the post-stack seismic traces and P-wave velocity models was approximated through this learning method. Before training the network, the seismic traces were transformed from the time domain to the depth domain based on the velocity models. Thus, the inputs and the outputs for training were in same domain. Most research has focused on identifying features and attributes in migrated images; few studies have discussed VMB or velocity inversion.
Multi-layer neural networks are computational learning architectures that propagate the input data across a sequence of linear operators and simple non-linearities. In this system, a deep convolutional neural network (CNN), proposed by LeCun et al. (2010), is implemented with linear convolutions followed by non-linear activation functions. A strong motivation to use FCN stems from the universal approximation theorem (Hornik, 1991;Csaji, 2001), which states that a feed-forward network with a single hidden layer containing a finite number of neurons can approximate any continuous function on compact subsets under a mild assumption on the activation functions. Additionally, FCN assumes that we learn representative features by convolutional kernels in a data-driven fashion to extract features automatically. Compared with DNN, FCN exhibits structures with fewer parameters to explain multi-layer perceptions while still providing good results (Burger et al., 2012).
In this study, we proposed the use of FCN to reconstruct subsurface parameters, i.e., P-wave velocity model, directly from raw seismic data, instead of performing a local-based inversion with respect to the subsurface represented through a grid. This method is an alternative formulation to conventional FWI and includes two processes. During the training process, multi-shot gathers are fed into the network together, and the network effectively approximates the non-linear mapping between the data and the corresponding velocity model.
During the prediction process, the trained network can be saved to obtain unknown geo-morphological structures only with new seismic data. Compared with traditional methods, less human intervention or no initial velocity models are involved throughout the process.
Although the training process is expensive, the cost of the prediction stage by the network is negligible once the training is completed. Alternatively, our proposed method provides a possible method for velocity inversion when the seismic data are in more realistic cases such as in the presence of noise and with a lack of low frequencies. Moreover, numerical experiments are used to demonstrate the applicability and feasibilities of our method. This paper is organized as follows. In the method section, we present a brief introduction to the basic inversion problem, the concepts of FCN, the mathematical framework, and the special architecture of the network. In the results section, we firstly show the dataset design. Two types of velocity models are discussed: a simulated dataset generated by the authors and an open experimental dataset of the Society of Exploration Geophysics (SEG). In addition, we compare the testing performance of our proposed method with that of conventional methods (i.e., FWI). In the discussion section, we present several open questions related to the utilization of our method for geophysical application. In the conclusion section, summaries of this study are presented, and future work is outlined. All of the acronyms used in this paper are listed in Table 1.

Basic inversion problem
The constant density 2D acoustic wave equation is expressed as where (x, z) denotes the spatial location, t represents time, v(x, z) is the velocity of the longitudinal wave at the corresponding location, u(x, z, t) is the wave amplitude, ∇ 2 (·) = ∂ 2 (·) ∂x 2 + ∂ 2 (·) ∂z 2 represents the Laplace operator, and s(x, z, t) is the source signal.
Equation 1 is usually given by where the operator H(·) maps v to u, and is usually nonlinear.
The classical inversion methods aim at minimizing the following objective function: where d denotes the measured seismic data, · 2 is the l 2 norm, and f (·) represents the data-fidelity residual.
In many applications, the engine for solving the above equation is to develop a fast and reasonably accurate inverse operator H −1 . An adjoint-state method (Plessix, 2006) is used to compute the gradient g(v) = ∇f (v), and iterative optimization algorithms are used to minimize the objective function. Owing to the nonlinear properties of the operator H and the imperfection of the surveys d, it is difficult to obtain precise subsurface models. Therefore, minimizing the above equation is generally an ill-posed problem, and the solutions are non-unique and unstable. If d contains full-waveform information, the above equation presents an FWI.
A review of the FCN Many DL algorithms are built with CNNs and provide state-of-the-art performance in challenging inverse problems such as image reconstruction (Schlemper et al., 2017), superresolution (Dong et al., 2016), X-ray-computed tomography (Jin et al., 2017), and compressive sensing (Adler et al., 2017). They are also studied as neuro-physiological models of vision (Anselmi et al., 2016).
The FCN, proposed by Long et al. (2015) in the context of image and semantic segmentation, changes the fully connected layers of the CNN into convolutional layers to achieve end-to-end learning. Figure 1 shows a sketch of a simple FCN. In this example, migrated seismic data are used as input, which is followed by a convolutional layer. Then, a pooling layer is inserted in the middle. After application of max-pooling, the sizes of the feature maps change to the previous one-half. Afterward, the transposed convolutional operation is applied to enlarge the size of the output to be the same as that of the input. Ultimately, we used a soft-max function to obtain the expected label, which is a label indicates which pixels belong to the salt structure in the migrated data. This FCN method can be described as follows: where N et(·) denotes an FCN-based network and also indicates the nonlinear mapping of the network, and x, y denotes the inputs and outputs of the network, respectively. Θ = {K 1 , K 2 , b 1 , b 2 } is the set of parameters to be learned, including the convolutional weights (K 1 and K 2 ) and the bias (b 1 and b 2 ). R(·) introduces the nonlinear active function, such as the rectified linear unit (Dahl et al., 2013), sigmoid, or exponential linear unit (Clevert et al., 2015). M (·) denotes the subsampling function (e.g., max-pooling, average pooling). * is the convolutional operation, and S(·) represents the soft-max function.

Mathematical framework
With the goal of estimating velocity models using seismic data as inputs directly, the network needs to project seismic data from the data domain (x, t) to the model domain (x, z), as shown in Figure 2. The basic concept of the proposed method is to establish the map between inputs and outputs, which can be expressed as where d is the raw unmigrated seismic data, and v denotes the P-wave velocity model predicted by the network. Our method contains two stages: the training process and the prediction process, as shown in Figure 3. Before the training stage, many velocity models are generated and are used as outputs. The supervised network needs pairs of datasets.
Therefore, the acoustic wave equation is applied as a forward model to generate the synthetic seismic data, which are used as inputs. Following the initial computation, the input-output pairs, which are named {d n , v n } N n=1 , are input to the network for learning the mapping.
During the training stage, the network learns to fit a nonlinear function from the input seismic data to the corresponding ground-truth velocity model. Therefore, the network learns by solving the optimization problem as where m represents the total number of pixels in one velocity model, and L(·) is a measure of the error between ground-truth values v n and prediction values v n . In our numerical experiments, the l 2 norm is applied for measuring the discrepancy.
For updating the learned parameters Θ, the optimization problem can be solved by using back propagation and stochastic gradient-descent algorithms (SGD) (Shamir and Zhang, 2013). The number of training datasets is large, and the numerical computation of the gradient Θ L(d; Θ) is not feasible based on our GPU memory. Therefore, to approximate the gradient, the mini-batch size h was applied for calculating L h , i.e., the error between the prediction values and the corresponding ground-truth values of a small subset of the whole training dataset, in each iteration. This led to the following optimization problem: Here, the ground-truth velocity models v n are given during the training process but are unknown during testing. One epoch is defined when an entire training dataset is passed forward and backward through the neural network once. The training dataset is first shuffled into a random order and is subsequently chosen sequentially in mini-batches to ensure one pass. It should be noted that the loss function is different from that (equation 3) in FWI, in which the loss measures the squared difference between the observed and simulated seismograms. In our case, we used the Adam algorithm (Kingma and Ba, 2014), i.e., a deformation of the conventional SGD algorithm. The parameters are iteratively updated as follows: where δ is the positive step size, and g( 1 mh Θ L h (d n ; Θ; v n )) denotes a function. This algorithm is straightforward to implement, computationally efficient, and well suited for large problems in terms of data or parameters.
The network is built once the training process is completed. During the prediction stage, other unknown velocity models are obtained by the available learned network. In our work, the input seismic data for prediction is also synthetic seismic traces. In a real situation, however, the input is field data. The method can be calculated by algorithm 1.

Architecture of the network
To achieve automatic seismic VMB from the raw seismic data, we adopted and modified the UNet (Ronneberger et al., 2015) architecture, which is a specific network built upon the concept of the FCN. Figure 4 shows the detailed architecture of the proposed network. It consists of a contracting path (left) used to capture the geological features and a symmetric shape of an expanding path (right) that enables precise localization. This symmetric form is an encoder-decoder structure and employing a contraction-expansion structure based on the max-pooling and the transposed convolution. The effective receptive field of the network increases as the input goes deeper into the network, when a fixed size convolutional kernel (3 * 3 in our case) is given. The numbers of channels in the left path are 64, 128, 256, 512, and 1024, as the network depth increases. Skip layers are adopted to combine the local, shallow feature maps in the right path with the global, deep feature maps in the left path.
2. Synthesize seismic data using the finite-difference scheme.
3. Input all data pairs into the network and use the Adam algorithm to update the parameters.
for t=1:1:T and (data, models) in traing set do for j=1:

2.Prediction process
1. Synthesize seismic data for different velocity models in the same way as that used for generating the training seismic data.
2. Input new seismic data into the learned network for prediction.
Output: Predicted velocity model v * were calculated per dimension over the mini-batch. ε is a value added to the denominator for numerical stability, γ and β are also learnable parameters; however, they were not used in our method.
We made two main modifications to the original UNet to fit the seismic VMB. First, the original UNet, proposed in the image-processing community, reads input images in RGB color channels that represent the information from the input images. To process the seismic data, we assigned different shot gathers, generated at different source locations but from the same model as channels for the input. Therefore, the number of input channels is the same as the number of sources for each model. The multi-shot seismic data were fed into the network together to improve data redundancy. Second, in a usual UNet, the outputs and inputs are in the same (image) domain. However, for our goal, we expected the network to realize the domain projection, i.e., to transform the data from the (x, t) domain to the (x, z) domain and to build the velocity model simultaneously. To complete this, the size of feature maps obtained by the final 3 * 3 convolution was truncated to be the same size as the velocity model, and the channel of the output layer was modified to 1. This was done so that the neural network could train itself during the contracting and expanding processes to map the seismic data to the exact velocity model directly. The main body of the network is similar to that in the original UNet, and 23 convolutional layers in total are used in the network.

NUMERICAL EXPERIMENTS AND RESULTS
In this section, the data preparation, including the model (output) design and data (input) design for training and testing datasets, is first presented. Subsequently, we use the simulated training dataset to train the network for velocity inversion, and we predict other unknown velocity models by the valuable learned network. Further, for SEG salt model training, the trained network for simulated models is regarded as the initialization; this pre-trained network is a common approach used in transfer learning (Pan and Yang, 2010).
The testing process on the SEG dataset is also performed. We compare the numerical results between our method and FWI. The numerical experiments are performed on an HP Z840 workstation with a Tesla K40 GPU, 32 Core Xeon CPU, 128 GB RAM, and an Ubuntu operating system that implements PyTorch (http://pytorch.org).

Data preparation
To train an efficient network, a suitable large-scale training set, i.e., input-output pairs, is needed. In a typical FCN model, training outputs are provided by some of the labeled images. In this paper, 2D synthetic models are utilized for testing the data-driven method.
Two types of velocity models are provided for numerical experiments: 2D simulated models and 2D SEG salt models extracted from a 3D salt model (https://wiki.seg.org/wiki/SEG/EAGE_3D_modelin Each velocity model is unique.

Training dataset
Model (output) design: To explore and prove the available capabilities of DL for seismic waveform inversion, we first generated random velocity models with smooth interface cur- Data (input) design: To solve the acoustic wave equation, we used the time-domain stagger-grid finite-difference scheme that adopts a second-order time direction and eighthorder space direction (Ozdenvar and McMechan, 1997;Hardi and Sanny, 2016). For each velocity model, 29 sources were evenly placed, and shot gathers were simulated sequentially.
The recording geometry consisted of 301 receivers evenly placed at a uniform spatial interval.
The detailed parameters for forward modeling are shown in Table 2. The perfectly matched layer (PML) (Komatitsch and Tromp, 2003) absorbing boundary condition was adopted to reduce unphysical reflection on the left, right, and bottom edges. Additionally, to verify the stability of our method, we added Gaussian noise, with zero mean and standard derivation of 5%, to each testing seismic data. Moreover, we made the amplitude of the seismic data two times higher. The noisy or magnified data were also used as inputs and were fed into the network to invert the velocity values.

Testing dataset
The ground-truth velocity models of the testing dataset had geological structures similar to those of the training dataset owing to the usage of supervised learning method. All of the velocity models for prediction were not included in the training dataset and were unknown in the prediction process. The input seismic data for prediction were also obtained by using the same method as that used for generating the inputs for the training dataset. For simulated models and SEG salt models, the testing dataset was composed of 100 and 10 velocity samples, respectively.

Inversion for simulated dataset
The first inversion case was performed for 2D simulated velocity models. During the training stage, the training batch for each epoch was constructed by randomly choosing 10 samples of velocity-model dimension 201 × 301 from the training dataset. In each pair data, the dimension of one-shot seismic data was downsampled to 400×301. The network deemed to work better was selected when the hyper-parameters were set as shown in Table 4 based on the training dataset and experimental guidance (Bengio, 2012). The mean squared error between the prediction velocity values and ground-truth velocity values is shown in Figure 9(a). Figure 10(j)- Figure 10(l) show three exemplified results of the proposed method. Visually, a generally good match was achieved between the predictions and the corresponding ground-truth.
In this case study, a comparison between our method and FWI was performed. We used the same parameter setting as that used to generate the training seismic data for the timedomain forward modeling. Multi-scale frequency-domain inversion strategy (Sirgue et al., 2008) was adopted. The selected inversion frequencies were 2.5 Hz, 5 Hz, 10 Hz, 15 Hz, and 21 Hz, based on the research of Sirgue and Pratt (2004). An adjoint-state based gradient descending method was adopted in this experiment (Plessix, 2006). The observed data of FWI was same as the seismic data we used for prediction. In addition, the true velocity model smoothed by the Gaussian smooth function was taken as the initial velocity model, as shown in Figure 10 To quantitatively analyze the accuracy of the predictions, we chose two horizontal positions, x = 900 m and x = 2000 m, and we plotted the prediction (blue), FWI (red), and ground-truth (green) velocity values in the velocity versus depth profiles shown in Figure   11. Most prediction values matched well with the ground-truth values. Moreover, in Figure   12, a comparison of the shot records of the 15th receiver is displayed, including the observed data using the ground-truth velocity model, reconstructed data obtained by simulating the inversion result of FWI, and reconstructed data obtained by forward modeling the prediction velocity model of our method. The reconstructed data with predictions obtained by the proposed method also matched well with the observed data.
The process of FWI for one simulated model inversion incurred a GPU time of 37 min.
In contrast, after training the FCN-based inversion network with 1078 min, the GPU time per prediction in our simulated models was only 2 s on the lower-equipment machine, which is more than 1000 times less than that for FWI.
To further validate the capability of the proposed method, more test results under more realistic conditions should be performed. Here, we present experiments in which seismic data are contaminated with random noise or the seismic amplitude is doubled.. The noisy data were generated by adding zero-mean Gaussian noise with a standard deviation of 5%. Similarly, to test the sensitivity to amplitude, another test was performed in which the amplitude of the testing seismic data was doubled; examples are shown in third column of Figure 14. In this test, the processed data were applied as the input for prediction; the performance comparison is displayed in Figure 13(d)- Figure 13(f). The prediction velocities using the processed inputs with higher amplitudes were consistent with the predictions using the original inputs. This is in compliance with the theoretical analysis and indicates that our proposed method achieves velocity inversion adaptively and stably.

Inversion for SEG salt dataset
To further show the outstanding ability of the proposed method, the trained network for the simulated dataset was utilized as the initialized network, which is one approach of transfer learning to train the SEG salt dataset. In the training process, the number of epochs was set to 50, and each epoch had 10 training samples (i.e., training mini-batch size). The other hyper-parameters for learning were same as those used for simulated model inversion. Figure 9(b) shows the mean-squared error of the SEG training dataset. The loss converged to zero when only 130 models were used for training. Similar to the test above, a comparison was performed between the proposed method and FWI with the same algorithm as that used in the experiments for simulated models. In this FWI experiment, the selected inversion frequency was 2.5 Hz, and the other three values ranged from 5 Hz to 15 Hz with a uniform frequency interval of 5 Hz. The initial velocity models were also obtained by using the Gaussian smooth function shown in Figure 15 Figure 16. In this test, compared with FWI method, the proposed method yielded a slightly lower performance, which could be attributed to the small number of training datasets. However, the predictions using the pre-trained initialized network (i.e., transfer learning in our study) were better than those obtained using the random initialized network. Moreover, the results of our method could serve as the initial models for FWI or travel time tomography. They could also be used for on-site quality control during seismic survey development.
The additional experiments using SEG salt datasets under more realistic conditions are also performed. When the seismic data were contaminated with noise, as shown in the second column of Figure 18, most prediction values obtained by the inversion network shown in Figure 17(a)- Figure 17(c) were close to ground-truth velocities, but were slightly lower than the predictions obtained using the clean data. However, the prediction results shown in Figure 17(d)- Figure 17(f) using the inputs with higher amplitudes, as shown in third column of Figure 18, made no differences. In this test, because the training dataset of the SEG salt models was less than that for the simulated models, the performance of the proposed method for the SEG dataset was not outstanding. Therefore, in future work, we will augment the diversity of the training set gradually and take advantage of the transfer learning to apply this novel method on other complex samples.  Table 5.
In summary, the numerical experiments provide promising evidence for the feasibility of our proposed method for velocity inversion from the raw input of seismic shot gathers directly without the need for initial velocity models. This indicates that the neural network can effectively approximate non-linear mapping even when the inputs have perturbation.

DISCUSSION
From an experimental perspective, the numerical results demonstrate that our proposed method presents promising capabilities of DL for velocity inversion. The objective of the research is to apply the latest breakthroughs in data science, particularly in DL techniques.
Although the indications of our method are inspiring, many factors can affect its perfor-mance, including the choice of training dataset; the selection of hyper-parameters such as learning rate, batch size, and training epoch; and the architecture of the neural network. For our purposes, we focused on the profound understanding of DL applied for seismic inversion.
Therefore a discussion on more impressive results and the advantages and disadvantages of the new method is provided in this section.
(a) How does the training dataset affect the network?
The limitation of our approach is that the capability of the network relies on the dataset.
In general, the models to be trained should involve structures or characteristics similar to those contained in the predictions. That is, the supervised learned network for prediction is limited to the choice of the training dataset, and the amount of data required for training depends on many factors. In most cases, a large amount of large-scale and diverse training samples results in a more powerful network. Moreover, the time consumed for training process is longer. One representative test of our proposed method for SEG salt models was conducted. In our experiments, predictions without salt are also presented. A comparison of the results between our method and FWI are shown in Figure 19. Our method yielded a lower performance than FWI. In particular, the sediment was vague because only 10 training samples without salt were utilized for training. Thus, the capability of the network to learn these models is lower than that for other salt models.
In addition, according to the simple geological structures contained in the simulated dataset, such as several smooth interfaces, increasing background velocity, and constantvelocity salt, relatively high similarity was noted between the training and testing datasets.
According to experimental guidance (Bengio, 2012)  In training stage of the DL method, one needs a training dataset that including lots of training pairs. The observed data could be a 1-shot gather, or other number shot gathers.
In our method, the shot numbers were fixed and were specific to the network such that 29 available shot seismic datasets were used for training the 29-shot network; in the prediction step, the same was considered for the 29 shot gathers. For further exploration, a comparison of performances with only 1-, 13-, 21-, 27-, and 29-shot training data is shown in Figure 20 owing to the constraint of the GPU memory. In Figure 20 the testing performance for the mean loss, mean peak signal-to-noise ratio (PSNR), and mean structural similarity (SSIM). The 1-shot case displayed a little bit unstable. The quantitative results may be misleading, however, because all testing evaluations are obtained for 10 selected network during training stage and take their average. In our next work, we will investigate the effects of training shots to apply the novel method in a more realistic scenario.
(b) How can we apply the network when a lack of low frequencies exists in the testing data?
The lack of low frequencies in field data is a main problem for practical application of FWI. However in the ML or DL methods, it is possible to learn the "low frequency" from simulation data or prior-information data. Two other numerical experiments are provided to show the performance of our method. As shown in Figure 21, all of the training datasets have low-frequency information, which is same as the original information used for prediction. However, the low frequency (i.e., 0-1/10 normalized Fourier spectrum) of the testing seismic data is removed by Fourier transform and the Butterworth high-pass filter.
Then, the reconstructed seismic data, shown in Figure 21(d), were used for prediction, the results of which are shown in Figure 21(b). In this case, the proposed method predicted most parts of the velocity model. A comparison of the prediction shown in Figure 21(a) with complete data shown in Figure 21(c) revealed that the structure boundaries are less clear, and the background velocity layers are somewhat vague, which could be attributed to the low-frequency information.
In addition, the performance of the supervised learning method relies on a training set. Therefore, the new training seismic dataset missing the low frequencies, which was processed by using the same approach as that used for the testing data (Figure 21(d)), and the corresponding ground-truth velocity models could be utilized to train the network. The prediction results are displayed in Figure 22. Visually, the predictions were slightly better than that shown in Figure 21 but were still lower than those with complete data.
(c) Is the learned network robust and stable for any prediction?
A general question often asked when learning is applied to some problems is whether the method can be generalized to other problems, e.g., whether a method trained on a specific dataset can be applied to another dataset without re-training. Thus far, it has been difficult to test complex (e.g., SEG salt models) or real models by using the trained network directly because the performance of our proposed method relies on the datasets, and similar distribution is relatively weak between the two types different velocity models. In our work, transfer learning (Pan and Yang, 2010), i.e., a research problem in ML that focuses on storing knowledge gained while solving one problem and applying it to a different but related problem, was applied when the new training models are similar to the simulated models.
The goal of using the pre-trained network as an initialization is to more effectively show the nonlinear mapping between the inputs and outputs rather than just allowing the machine to remember the characteristics of the dataset. A comparison of the training loss versus the number of epochs between random initial networks (i.e., the same as parameter initialization in UNet) and a pre-trained initial network (i.e., trained network for a simulated dataset) is shown in Figure 23. The network learned better with the pre-trained initialization in the same computational time.

CONCLUSION
In this study, we proposed a supervised end-to-end DL method in a new fashion for velocity inversion that presents an alternative to "conventional" FWI formulation. In the proposed formulation, rather than performing local-based inversion with respect to subsurface parameters, we used a FCN to reconstruct these parameters. After a training process, the network is able to propose a subsurface model from only seismic data.  Table 3. 12 Shot records of the 15th receiver. Given from left to right in each row are the observed data according to the ground-truth velocity model (shown in Figure 10(a)- Figure   10(c)), reconstructed data by obtained by forward modeling of the FWI inverted velocity model (shown in Figure 10(g)- Figure 10(i)),and reconstructed data obtained by forward modeling the prediction of the FCN-based inversion method (shown in Figure 10(j)- Figure   10(l)).
13 Sensitivity of the proposed method to noise and amplitude (simulated models): (a)-(c) prediction results with the noisy seismic data; (d)-(f) prediction results with magnified seismic amplitude. Our method showed acceptable results when the input data were perturbed.
14 Comparison of records of simulated seismic data. Given in each row from left to right are original data, noisy data (with added Gaussian noise), and magnified data (to twice as large). The corresponding velocity models of each row are the three models shown in Figure 10 Table 3. Our method showed a slightly lower performance than FW because only 10 training models without the salt body were utilized. More training data are required to obtain corresponding improvement.