{'fraction_non_alphanumeric': 0.03341625207296849, 'fraction_numerical': 0.01318407960199005, 'mean_word_length': 4.863393291200778, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 1, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Machine Learning Regularized Solution of the Lippmann-Schwinger Equation

Solution of the discretized Lippmann-Schwinger equation in the spatial frequency domain involves the inversion of a linear operator specified by the scattering potential. To regularize this inevitably ill-conditioned problem, we propose a machine learning approach: a recurrent neural network with long short-term memory (LSTM) and with the null space projection of the Lippmann-Schwinger kernel on the recurrence path. The learning method is trained using examples of typical scattering potentials and their corresponding scattered fields. We test the proposed method in two cases: electromagnetic scattering by dielectric objects, and electron scattering by multiple screened Coulomb potentials. In both cases the solutions to test examples, disjoint from the training set, were obtained with fewer iterations and were accurate compared to linear solvers. We also observed surprising generalization ability: in the electromagnetic case, an LSTM trained with random arrangements of dielectric spheres was able to obtain the correct solutions for general topologically similar objects, such as polygons. This suggests that the LSTM successfully incorporates the physics of scattering into the inversion algorithm.

The Fredholm integral equation of the second kind, in physics more commonly known as Lippmann-Schwinger equation (LSE) ψ(r) = ψ i (r) + G (r − r ) V (r ) ψ (r ) dr , (1) results from several classical and quantum scattering principles. It relates the scattered field ψ to the incident field ψ i , scattering potential V and free-space Green's function G. After adequate spatial sampling of ψ to a discrete wavefunction x, Eq. (1) may be converted to a linear inverse problem [1][2][3] Similar linear problems result from numerous partial differential equations and integral equations [4,5] and extensive research has been devoted to linear system solvers [6,7]. Since the late 1990s, the solution of linear inverse problems has been increasingly informed by the branch of function approximation theory that deals with sparse representations [8]. Sparsification of the wavefunction sometimes is appropriately ad hoc, e.g. using redundant bases such as wavelets [9]; whereas at other times it is best learnt from data. This last approach requires several examples of typical solutions x and their corresponding "measurements" b to be available, and the resulting algorithms are referred to as dictionaries [10,11]. The inverse estimate is obtained aŝ where Φ(x) is the sparsity-promoting regularizer, and κ is the regularization parameter. Typical such regularizers include the L 1 norm, i.e. are non-differentiable. The solution of Eq. (3) proceeds iteratively in computational (discrete) time t according to the proximal gradient method [12] aŝ or one of its speedier relatives such as FISTA [13]. The proximal operator P Φ is derived from the regularizer Φ. The second term in the bracket is the null space projection, and it can be thought of as encoding the physical model A into the learning scheme. In 2016, it was proposed that, instead of using the L 1 norm regularizer κ chosen in an ad hoc manner and then running Eq. (4), sparsifiers even better fit to available data {x, b} may be obtained by learning P Φ which is represented by a linear combination of basis functions [14]. Extending the space of functions that can be learnt, a neural network is used to approximate P Φ [15]. This is schematically represented in Fig. 1. However, implementations of this idea to actual linear inverse problems Eq. (2) soon realized that the recurrence destabilizes the neural network's training [16]. Thus, they resorted to breaking the recurrence into a cascade, occasionally trained in adversarial fashion [15]. The cascade means that now multiple neural networks need to be trained; and the number of recurrences allowed needs to be builtin the architecture irrevocably. Here, we acknowledge that recurrent dynamics, as in Fig. 1, are stabilized if one replaces the neural network with a long short-term memory (LSTM) C LSTM whose key function is selective temporal forgetting [17].
Thus, to solve (2) we apply the recurrencê We find that, applied to scattering problems, the recurrent method Eq. conjugate gradient method (CG); and generalizes surprisingly well to scattering potentials significantly deviating from the training set. We attribute this latter property to the physical operator A becoming strongly ingrained into the LSTM through the recurrent learning process. The approach of Eq. (5) for solving (2) may be applicable for more general physical systems; we chose here the LSE, as an especially acute case that becomes ill-conditioned as the frequency increases and when multiple scattering cannot be neglected [2,3].
We perform a numerical experiment to test the applicability of Eq. (5) to solve physical problems of type Eq. (2). As an example, the scattering of the incident wave ψ i (r) = e ikz , where λ = 2π/k is the wavelength, is studied. We consider two types of the potential V . First, which describes the optical scattering in isotropic and linear media with refractive index n; and second, which represents screened Coulomb potential on an electron from P unit charges with a screening constant ξ. The error function erf with a parameter η is used to regularize the singularities at r j without severely compromising physical accuracy [18]. For the numerical evaluation of the LSE, we consider a cubic domain On Ω, the LSE is discretized as Eq. (2) by expanding ψ and ψ i in the Fourier basis and substituting the integral with the discrete convolution [1,19]. More information on the numerical implementation is provided in the Supplemental Material. Eq. (2) is iteratively solved with the CG until the relative L 2 -norm error ψ − ψ i 2 / ψ i 2 reaches 10 −6 . C LSTM is trained for each type of potential. 9600 pairs of (ψ, V ) are generated by the CG and split into 9000 training, 300 validation, and 300 test examples. For the optical scattering problem, k, L, and N are 2π/λ, 12, and 64, respectively where λ, the wavelength, is 0.532. The refractive index n consists of 1 to 3 spherical objects whose magnitude ranges from 1.01 to 1.03 and diameter  , the structural similarity index (SSIM) [21], and the peak signal-to-noise ratio (PSNR). We assess not only the phase and the amplitude of complex fields but the phase unwrapped along the optical axis (z) to prevent any artifacts arising from the phase discontinuity.
Numerical study on optical scattering.-To test the validity of the CG solutions, they are compared to the Mie theory and the finite-difference time-domain method in the Supplemental Material. In overall, the LSE solved with the CG can well estimate the Mie and finitedifference time-domain solutions with tolerable errors.
The scattered fields generated by the LSTM are shown in Fig. 2. As can be seen in Figs. 2(a)-(c), the LSTM results well approximate the CG solutions with a slight loss of high-frequency features in the amplitude, while retaining major interference patterns. Table I summarizes the image quality metrics for the 300 test samples, which shows that the relative error per voxel is less than 0.01 for both the amplitude and the phase. Achieving such low error implies that the LSTM has successfully learnt the appropriate regularization for the spherical objects. In the Supplemental Material, the LSTM results are compared with those from the U-Net [22], one of the stateof-art neural network architectures used for image segmentation and image regression tasks [23,24]. Though it leverages significantly larger number of learnable parameters and its estimations exhibit the similar image quality metric values, the U-Net produces major visual artifacts, which suggests that including the physics of the system A makes the training process more efficient.
Interestingly, if we pass test examples with shapes completely different to the train examples, e.g. complex polyhedra, the LSTM still appears to well approximate the CG solution, as shown in Fig. 3. The LSTM results in Fig. 3 suffer from more visual artifacts at which the incident field first contacts the tetrahedral object, compared to other regions. Such artifacts can be attributed to the difference between the surfaces of the objects in the training examples and polyhedral objects. In other words, polyhedral objects have relatively sharp edges and vertices that cannot be experienced during the training, which may be able to worsen the estimation quality. In the Supplemental Material, we test the LSTM with 5 objects of polyhedral shapes other than the tetrahedron, which also displays the similar behavior. Nonetheless, the overall estimation retains physically reasonable quality. As listed in Table II, the image quality metrics for such polyhedral objects also show tolerable values. The overall agreement between the results from the CG and the LSTM indirectly indicates that the physical information from A is properly fused into the training process and the LSTM layer acts as an actual proximal operator to increase the convergence of the iterative algorithm. Otherwise, e.g. if the LSTM is merely overfitted to the spherical training examples, the LSTM would map the output of the physics layer to a wrong space, leading to a poor convergence.
Numerical study on Coulomb scattering.-The validity of the CG method on Coulomb potential is tested with an analytic solution. For this test, we exclude the screening and the singularity regularization, i.e. V (r) = 2/|r| is replaced by (7). Detailed information on the computational architecture and the result is presented in the Supplemental Material. The comparison shows that the LSE solved with the CG well approximates the interference near the center of the electron beam and the numerical difference to the analytical solution is tolerable, implying that the CG may be applicable to estimate Coulomb scattering.
Coulomb scattering expected by the LSTM is shown in Fig. 4. As in the optical scattering case, visually the LSTM approximates the CG solutions with small deviation. The region that exhibits somewhat high relative error is in the vicinity of the two nuclei where the potentials are highly overlapping. Since the potential profile becomes deep and its shape complex, this implies that it is difficult for the LSTM to learn the regularizer. Table III outlines the numerical assessment of the LSTM performance. Among the 300 test samples, the relative error per voxel is comparable to 1.0 × 10 −3 , which, combined with the high SSIM and the PSNR values, means that the LSTM can be utilized to estimate the solution of the LSE not only for the optical potentials but the quantum mechanical potentials.
Conclusion.-Numerous physical systems have been formulated as the linear inverse problem and there have been corresponding studies to solve Eq. (2), including not only classical optimization methods but data-driven approaches and the neural networks [25][26][27]. Compared to the previous studies, we tailor the performance of the neural networks by explicitly informing the physics of the system A. Furthermore, our LSTM generalizes well to unseen examples by explicitly learn the regularizating prior underlying the specific scattering matrix A. Hence, it can be expected that our work can contribute to solving physical problems other than the LSE. Recently, an alternative to LSTM known as Gated Recurrent Unit (GRU) was proposed [28] and shown to generalize better in certain machine vision and robotic manipulation problems. Even though we did not investigate GRU in this study, it certainly shows merit for future investigations [28]. This work is supported by the Intelligence Advanced Research Projects Activity (IARPA) No. FA8650-17-C-9113 and Singapore-MIT Alliance for Research and Technology (SMART) Grant No. 015824-00169. * sbpang@mit.edu † Also at: Singapore-MIT Alliance for Research and Technology (SMART) Centre, 1 CREATE Way, Singapore