{'fraction_non_alphanumeric': 0.03020656479909451, 'fraction_numerical': 0.013405489530277306, 'mean_word_length': 4.908672936259143, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Determining cross sections from transport coefficients using deep neural networks

We present a neural network for the solution of the inverse swarm problem of deriving cross sections from swarm transport data. To account for the uncertainty inherent to this somewhat ill-posed inverse problem, we train the neural network using cross sections from the LXCat project, paired with associated transport coefficients found by the numerical solution of Boltzmann's equation. The use of experimentally measured and theoretically calculated cross sections for training encourages the network to avoid unphysical solutions, such as those containing spurious energy-dependent oscillations. We successfully apply this machine learning approach to simulated swarm data for electron transport in helium, separately determining its elastic momentum transfer and ionisation cross sections to within an accuracy of 4% over the range of energies considered. Our attempt to extend our method to argon was less successful, although the reason for that observation is well-understood. Finally, we explore the feasibility of simultaneously determining the cross sections for multiple scattering processes using this approach.


Introduction
Control and optimisation of plasma processing is dependent on accurate modelling of the associated charged particle transport. Fundamental to this is the provision of complete and accurate sets of cross sections. These cross sections must have embedded particle, momentum and energy balance built into them. Experiment (e.g. crossed beam [1]) and theory (e.g. convergent close-coupling [2][3][4][5]) can provide some of this required information however there is often still uncertainty associated with these cross-sections (e.g. where extrapolation is required, or where theories break down) and here "educated guesses" are often used. Swarm experiments provide accurate yet independent data to assess the self-consistency of these cross sections [6], which in principle makes them very useful to evaluate their utility.
In this study, we assess the inference/extraction of microscopic information from measured swarm data. The first attempts at deriving electron scattering cross sections from swarm transport coefficients were made in the 1920s by Mayer [7], Ramsauer [8] and Townsend et al. [9]. Early approaches such as these assumed a simplified form of the electron energy distribution function (EEDF), such as a Maxwellian or Druyvesteyn distribution, from which theoretical transport coefficients could be calculated and contrasted with those measured experimentally. By iteratively adjusting the cross sections until the calculated transport coefficients coincided with those from experiment, a plausible solution to this inverse swarm problem could be found. The accuracy of this iterative approach was improved in the 1960s when Phelps, alongside numerous collaborators, began determining the EEDF from the numerical solution of Boltzmann's equation [10][11][12][13][14]: where f (t, r, v) is the phase-space distribution function of the electron swarm, e is the elementary charge, m is the electron mass, E is the applied electric field, and ∂f ∂t coll contains operators for all the considered electron collision processes.
When the available swarm transport data is limited, the inverse swarm problem becomes illposed with solutions that are non-unique and that consist of cross sections that are sensitive to small variations in the transport coefficients, as illustrated in Figure 1. In such situations this lack of information can to some extent be made up for by the experience and intuition of an expert. Due to this required expertise, as well as the tedious nature of this trial and error approach to swarm analysis, a number of automated methods for solving the inverse swarm problem have been proposed [15][16][17][18][19][20][21]. Of particular relevance to the present work is an investigation in the early 1990s by Morgan [19], who solved the inverse swarm problem using an artificial neural network that had been trained on exemplar mappings between swarm data and cross sections. He concluded that such a machine learning approach was well-suited for obtaining an approximate cross section, that could subsequently be refined further by nonlinear least-squares optimisation. Since Morgan's pioneering investigation, there have been major developments in the field of machine learning that have allowed for much larger and more powerful models to be realised [22][23][24][25][26][27]. Given that there is now also a wealth of cross section data, made available by the LXCat project [28][29][30], this data-driven approach to the inverse swarm problem warrants renewed investigation. The remainder of this paper is structured as follows. In the following section, we present a new neural network for the solution of the inverse swarm problem. In Section 3, we apply this neural network to determine cross sections of helium and argon using simulated swarm transport data. Finally, Section 4 presents some conclusions and a discussion of future generalisations and Figure 1. Euler diagram illustrating the somewhat ill-posed nature of the inverse swarm problem. Arrows denote the forward problem of mapping from a set of cross sections to a set of corresponding transport coefficients. This mapping is well-posed and can be achieved numerically by solving Boltzmann's equation (1) or by performing a Monte Carlo simulation, among other techniques. Conversely, the inverse problem is ill-posed due to its potentially ambiguous solutions and its sensitivity to the transport data. By training the neural network, Eq. (9) on cross sections from the LXCat project, we restrict the solution to a subset of physically-plausible cross sections, thereby improving the posedness of the inverse problem.
applications of this work.

Neural network regression of cross sections
Artificial neural networks are highly-parameterised mathematical functions capable of universal function approximation [31,32]. By carefully adjusting the parameters of a neural network, in a process called training, it is possible to approximate arbitrary multivariate vector-valued functions. In other words, neural networks are capable of performing nonlinear mappings between vector spaces. In this section, we present a neural network for the solution of the inverse swarm problem that learns from the inverse of the forward mapping from cross sections to transport coefficients. This process is schematically depicted in Figure 1. Our neural network is of a similar architecture to that employed by Morgan [19] and, as such, we will begin by outlining the structure of Morgan's neural network before describing how ours differs.

Model introduction and overview
To perform the cross section regression given the transport coefficients, we use a fully-connected neural network (FCNN). The simplest FCNN is an affine transformation of an input vector x to an output vector y: where the matrix W and vector b contain the network parameters. More generally, to describe nonlinear relationships, additional affine transformations are applied, each interleaved with a nonlinear activation function. For example, Morgan considered an FCNN of the form [19]: where a hyperbolic tangent activation function is applied element-wise throughout, and each affine transformation A n (x) = W n x + b n has associated parameters W n and b n . This network is said to have 4 layers of neurons, corresponding to the vectors of numbers reached at each stage of the computation. For example, the above network has an input layer x, an output layer y and 2 intermediate hidden layers x 1 and x 2 , which arise as follows: While the input and output layers, x and y, are generally of a size fixed by the nature of the problem at hand, the number and size of the hidden layers must be specified. With too few hidden neurons, the predictive power of the model is hindered. Conversely, a model that is too complex can overspecialise to the training data, resulting in poor generalisation to gases outside of the training dataset. Fortunately, this latter problem becomes less likely to occur as the amount of training data increases. The main architectural difference between our network and Morgan's is the structure of the input and output vectors, x and y. Morgan considered an input vector x containing measured drift velocities W and characteristic energies D T /µ, where D T is the bulk transverse diffusion coefficient and µ is the bulk electron mobility, and an output vector y as a discrete representation of the cross section of interest, σ (ε) [19]: Here, the input transport coefficient measurements are performed at various reduced electric fields E 1 /n 0 , E 2 /n 0 , . . ., where n 0 is the number density of the background neutrals, and the output cross section is sampled at discrete energies ε 1 , ε 2 , . . .. As input in our case, in addition to the transport coefficients of bulk drift velocity W and bulk longitudinal diffusion n 0 D L , we also consider the first Townsend ionisation coefficient α/n 0 . For the output cross section, rather than discretising as above, we use the neural network itself as a discrete approximation of the cross section as a function of energy. Energy, ε, now becomes an input to the neural network alongside the transport coefficients, and the output is now a single number corresponding to the cross section evaluated at this energy, σ (ε): Although it is not shown here, before being used for regression all the data is log-transformed with Eq. (12). This is detailed in the following section. This architectural change of making the energy an explicit input, illustrated in Figure 2, provides a number of benefits. Firstly, rather than having to decide on a one-size-fits-all cross section discretisation, the neural network learns a suitable discretisation that is tailored to the training data. Secondly, now that there is only one output neuron per cross section, far fewer weights are required in the final affine transformation, making the network simpler to train, faster to evaluate, and less likely to overspecialise to the training data. This is most beneficial when multiple output cross sections are considered, such as in Section 3.3. Finally, it is now relatively trivial to constrain the cross section training data when there are known upper and lower bounds obtained from, for example, crossed beam experiments [1]. This is awkward to do when simulatenously predicting a cross section at multiple energies, as in Eq. (7), because then constraints must also be enforced simulatenously for every energy considered.

Data preparation and model training
To train the neural network, Eq. (9), we require example inputs and outputs, a cost function that indicates the performance of the neural network on this training data, and an optimisation algorithm to minimise this cost function by varying the parameters of the neural network. We consider first the issue of determining suitable cross sections for the training. For the purpose of fitting elastic momentum transfer cross sections, Morgan generated training examples using a power-law model of the form σ (ε) ∝ ε p , where −1 ≤ p ≤ 1 [19]. Similar options exist for  other processes, e.g. Machacek et al. described the cross section for total positronium formation by positron scattering using a surge function σ (ε) ∝ ε p e −bε , where p and b are shape parameters [34]. Parameterised cross sections such as these are ideal for machine learning, as they can be sampled from indefinitely for training data. That said, we choose instead to train using a finite set of cross sections from the LXCat project [28][29][30], so as to expose the neural network to as much measured and calculated cross section physics as possible. To make up for this finite amount of cross section data, we generate training data by random interpolation of LXCat cross sections. Specifically, given a unique pair of cross sections, σ 1 (ε) and σ 2 (ε), as well as a uniformly distributed mixing ratio r ∈ [0, 1], we form each training cross section by evaluating: where ε 1 and ε 2 are the respective threshold energies of σ 1 (ε) and σ 2 (ε). This is a weighted geometric average of the two cross sections that has been shifted to have a new threshold of ε 1−r 1 ε r 2 , which is itself a weighted geometric average of the separate threshold energies. We use the same ratio r when mixing both cross sections and threshold energies, as there is a negative correlation between the magnitude of a cross section and its threshold energy, depicted in Figure 3, that we would like to see reflected in the training data.
Each cross section generated using Eq. (10) must be sampled at discrete energies in order to be used for training. To aid the neural network in learning the global energy-dependence of the cross sections, we sample at a large number of energies over the domain of interest. These energies are also selected geometrically: where [ε min , ε max ] is the energy domain under consideration and s ∈ [0, 1] is a uniformly distributed random number. Finally, from our chosen cross section or cross sections, we must determine corresponding transport coefficients to complete our input/output training pair. For efficient and robust generation of this swarm data, here we apply the two-term approximation [35,36] to Boltzmann's equation (1) and then perform backward prolongation [37] of the EEDF by inward integration from high to low energies. This integration is performed numerically, using an adaptive order adaptive energy Adams-Moulton method [38], as implemented in the DifferentialEquations.jl software ecosystem [39][40][41].
Since cross sections, energies and transport coefficients all span many orders of magnitude, we make sure to log-transform everything before training. That is, given a strictly positive quantity z, we apply a log-transformation that is scaled and shifted to ensure all the training data lies within the domain [−1, 1]: where z min and z max are the extrema of all instances of z in the training dataset. If any instance of z happens to be zero, we replace it with a suitably small positive number before applying the above transformation.
To train the neural network, we minimise its mean absolute error on the training set of cross sections: where the index i ranges over the entire set of N training examples (x i , y i ), and σ (x i ) is the corresponding neural network prediction for each. We have purposefully avoided using the mean squared error, as that encourages the neural network to preferentially improve upon its worst predictions. This sounds reasonable, and is the motivation behind least-squares regression, but as we are solving an inverse problem there is an inherent uncertainty to the solution that fundamentally limits how well the network can perform. Encouraging the neural network to do the impossible of accurately and consistently fitting the least certain cross section values is expected to hinder its performance overall. We implement and train the model using the Flux.jl machine learning framework [42]. In constructing the neural network, we initialise the parameters in b n to zero and those in W n to uniform random numbers as described by Glorot and Bengio [43]. Then, we train by minimising Eq. (13) using the Adam optimiser [44] with step size α = 10 −3 and exponential decay rates β 1 = 0.9 and β 2 = 0.999. By definition, training improves the performance of the neural network on the training examples, with the hope that good performance on the training set will be correlated with good performance in general. However, with too much training there lies a danger of the neural network learning features that are unique to the training data, at the expense of generalisation [45]. A common technique to avoid this is to simply stop the training before the network's ability to generalise worsens, as quantified by some other measure independent of the data used for training [46]. For this measure, we use the necessary condition that the solution to the inverse swarm problem must also satisfy the forward swarm problem. Thus, we continually apply the Boltzmann solver, while training, to determine how well the fitted cross sections reproduce the transport coefficients that were used to perform the fit. We then stop training when the mean squared error in this reproduced swarm data reaches a minimum, as demonstrated in Figure 4.   Figure 7, alongside the mean squared error (green) in reproducing the transport coefficients used to perform the fit. As training progresses, the error for the training dataset decreases as expected. Beyond a certain point, however, the neural network overspecialises to the training data, at the expense of the argon fit. We conclude that the ideal place to stop training is where the mean squared error in the transport coefficients is minimised, indicated here by the vertical dashed line.

Application to simulated swarm data
In this section, we simulate pulsed Townsend swarm experiments for electron transport in helium and argon at a temperature T = 300K, across a range of 50 reduced electric fields E/n 0 spaced exponentially between 10 −3 Td and 10 3 Td inclusive, where 1Td = 1 Townsend = 10 −21 Vm 2 . Using the resulting transport coefficients of bulk drift velocity W , bulk longitudinal diffusion coefficient n 0 D L and Townsend coefficient α/n 0 as inputs, we then apply the neural network Eq. (9) toward fitting various cross sections of helium and argon over the energy domain 10 −1 eV, 10 2 eV . For both gases, cross section data are sourced from the Biagi v7.1 database [47].

Fitting elastic momentum transfer cross sections
We consider first fitting helium's elastic momentum transfer cross section (MTCS), while assuming its inelastic cross sections are known. For the training data, we use Eq. (10) to randomly sample 10 4 elastic MTCS from those plotted in Figure 5 from LXCat [2][3][4][5]. Of course, helium is excluded when performing this sampling. For each of these generated cross sections, corresponding swarm transport coefficients are found. Finally, each cross section is sampled at 100 random energies between 10 −1 eV and 10 2 eV, using Eq. (11), resulting in a total of 10 6 training examples. We split these training exemplars into 10 batches of 10 5 examples each, making sure to shuffle the training data beforehand so that each batch is representative of the training set as a whole. Training then proceeds iteratively by repeatedly cycling through each batch in turn, updating the neural network  [2][3][4][5]. These are randomly sampled from using Eq. (10) and the resulting cross sections are employed in the training of the neural network depicted in Figure 2 for the purpose of fitting elastic momentum transfer cross sections. A similar approach is taken when fitting cross sections for other collision processes.
parameters each time using the optimiser. Training is continued until the mean squared error in the resulting transport coefficients reaches a minimum, as demonstrated in Figure 4. The final result of this process, for the fitted cross section and corresponding transport coefficients, is plotted in Figure 6. The neural network is found to be accurate here to within 4%, for both the elastic MTCS and its corresponding transport coefficients.
We now repeat the above process for argon (whose elastic MTCS is now excluded from the training data instead of helium's) and plot the results in Figure 7. Unlike helium, argon's elastic MTCS is an outlier among the training data, with its Ramsauer-Townsend minimum dipping below the rest of the LXCat cross sections in Figure 5. Because of this, the neural network struggles to determine the correct magnitude of that minimum, believing it to be more than twice as large as it should. In effect, our choice of training data has incorrectly constrained the fitted cross section. The error at larger energies, above 0.4eV, is however fortunately considerably better, lying to within 20%. This larger error, as compared to helium, is likely due in part to the neural network compensating for the incorrect Ramsauer-Townsend minimum, so as to better match the transport coefficients. Lastly, it should be noted that a larger error is to be expected for argon. This follows as the chosen swarm data provides less information about argon's elastic MTCS than it does for helium's, as evidenced by the mean energy of electrons at the highest applied field, E/n 0 = 1000Td, which is ∼ 133eV for helium and only ∼ 15eV for argon. Figure 6. Neural network regression of helium's elastic MTCS, alongside corresponding plots of the transport coefficients. The neural network determines the cross section to within an accuracy of 4% here, while also being consistent with the provided swarm data to within the same margin of error. Figure 7. Neural network regression of argon's elastic MTCS, alongside corresponding plots of the transport coefficients. Due to the uniqueness of argon's Ramsauer-Townsend minimum, the training data, plotted in Figure 5, is insufficient to perform an accurate cross section fit. Away from that minimum, the fit is accurate to within 20%. It is expected that the fit quality would improve if more suitable cross sections were to be available and thus used for training.

Fitting threshold cross sections
We turn now to fitting the ionisation cross section of helium, as a representative example of data with a clear threshold energy. We generate our training data using LXCat as before, only now employing the ionisation cross sections instead [2-5, 47-52, 55-71]. Fortunately, besides from this change in the utilised cross section data, no other alterations are required to the neural network or training procedure in order to fit a threshold cross section. Figure 8 plots the result of the neural network regression, alongside corresponding transport coefficients. Above threshold, the fit of helium's ionisation cross section is in error by at most 4%, hence on a par with the previous fit of helium's elastic MTCS. It is very promising to see that the neural network is able to successfully identify the ionisation threshold from the transport data. Machine learning approaches thus have the potential for determining unknown threshold energies, such as the neutral dissociation threshold of tetrahydrofurfuryl alcohol (THFA), a molecule of biological interest [72][73][74][75][76][77][78][79].

Fitting multiple cross sections simultaneously
Finally, we consider simultaneously fitting helium's elastic momentum transfer, n = 2 singlet excitation (2 1 S and 2 1 P ), n = 2 triplet excitation (2 3 S and 2 3 P ) and ionisation cross sections. Accordingly, we now also incorporate excitation cross section data from LXCat into the training procedure [2-5, 47-52, 54-59, 61-66, 68, 70, 71, 80]. As in the previous fits, we assume nothing here but the number and type of each cross section. Correspondingly, the output layer of our neural network is now increased to contain 4 neurons, one for each cross section being fitted. How these cross sections are ordered in the output layer does not matter, so long as the ordering is kept consistent when training and applying the network. With this in mind, note that it is not clear how to consistently order each training pair of n = 2 excitation cross sections. This is because excitation processes can be interchanged with one another without any affect on the transport coefficients. If we are not careful, the neural network could proceed to correctly predict the excitation cross sections of helium, but be penalised for doing so because the cost function, Eq. (13), happens to assume the opposite ordering. As such, for this fit, we modify the cost function so as to make it symmetric with respect to the excitation cross sections. In effect, for each training exemplar, we try both permutations of predicted excitation cross sections and select the one that minimises the cost function the most. Figure 9 plots the result of the neural network regression for this case. Here we observe the fitted elastic MTCS to be accurate to within roughly 10% over the entire range of energies. This is an expected increase over the maximum 4% error observed in Section 3.1, where the elastic MTCS was fitted exclusively. For inelastic processes above threshold, we see larger cross section errors of within 20% for total n = 2 excitation and 25% for ionisation. Of the pair of n = 2 excitation cross sections determined by the network, one was comparable to the total n = 2 excitation cross section, while the other was made very small. This inability of the network to unfold the separate n = 2 singlet and n = 2 triplet excitation cross sections is likely due in part to their very similar threshold energies. Overall, it is expected that the ability of the neural network to accurately unfold multiple cross sections would improve with the consideration of additional swarm data, such as transport coefficients of helium mixed with other gases.

Conclusion
We have presented a machine learning approach to the inverse swarm problem where a neural network is used to approximate unknown cross sections as a functions of energy, given corresponding swarm transport data. By training this network on physical cross sections from the LXCat project [28][29][30], it was found to yield physically-plausible solutions to the inverse swarm problem that are consistent with the specified transport coefficients. As a demonstration, we applied this network to determine cross sections of helium and argon from simulated pulsed Townsend electron swarm data Figure 9. Neural network regression of helium's elastic momentum transfer, n = 2 singlet excitation, n = 2 triplet excitation and ionisation cross sections, alongside corresponding plots of the transport coefficients. As expected, the overall error has increased here compared to fitting helium's elastic momentum transfer and ionisation cross sections individually. The network was unable to unfold the separate cross sections for n = 2 singlet and triplet excitation, although the values it predicts are consistent with the threshold and magnitude of the total n = 2 excitation cross section. over a range of reduced electric fields E/n 0 . We found that a suitably-trained neural network could determine individual elastic momentum transfer and ionisation cross sections to within 4% accuracy for the swarm data considered. From the same swarm data, we were also able to simulatenously fit the elastic MTCS, total excitation, and ionisation cross sections of helium to within 10%, 20% and 25% accuracy respectively. Promisingly, this data-driven approach to swarm analysis not only avoids the tedium of conventional iteration, but also appears to provide an adequate substitute for the intuition of an expert.
A fundamental limitation of our neural network is that it provides only a single solution to a problem for which many plausible solutions may exist. Ideally, we would like to be able to quantify this uncertainty in the solution and place corresponding error bars on the predicted cross sections. One such approach involves determining a probability distribution of cross sections, from which plausible solutions can be sampled. This process is known as conditional density estimation and a number of machine learning models are capable of performing it, including mixture density networks [81], conditional variational autoencoders [82], conditional generative adversarial networks [83] and conditional flow-based generative models [84][85][86]. Work on addressing this present limitation in our approach is currently underway.
Finally, before our neural network can be applied to actual swarm measurements, the presence of experimental error in these measurements must be accounted for. Fortunately, by simply introducing simulated experimental error estimates in the transport coefficients used for the training, we should expect the neural network to adapt accordingly. Indeed, this is the principle behind neural network approaches to noise reduction in images [87]. This issue will be addressed in our future studies.