{'fraction_non_alphanumeric': 0.02923761415716054, 'fraction_numerical': 0.0075268457498411, 'mean_word_length': 4.546196660482375, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Semi-intrusive uncertainty propagation for multiscale models

A family of semi-intrusive uncertainty propagation (UP) methods for multiscale models is introduced. The methods are semi-intrusive in the sense that inspection of the model is limited up to the level of the single scale systems, and viewing these single scale components as black-boxes. The goal is to estimate uncertainty in the result of multiscale models at a reduced amount of time as compared to black-box Monte Carlo (MC). In the resulting semi-intrusive MC method, the required number of samples of an expensive single scale model is minimized in order to reduce the execution time for the overall UP. In the metamodeling approach, the expensive model component is replaced completely by a computationally much cheaper surrogate model. These semi-intrusive algorithms have been tested on two case studies based on reaction-diffusion dynamics. The results demonstrate that the proposed semi-intrusive methods can result in a significant reduction of the computational time for multiscale UP, while still computing accurately the estimates of uncertainties. The semi-intrusive methods can, therefore, be a valid alternative, when uncertainties of a multiscale model cannot be estimated by the black-box MC methods in a feasible amount of time.


Introduction
Computer modeling is widely used in science and engineering to study systems of interest and to predict their behaviour. These systems are usually multiscale in nature, as their accuracy and reliability depend on the correct representation of processes taking place on several length and time scales [9,12,14,17,23]. Moreover, these multiscale systems are usually stochastic, since there are always some unresolved scales, whose effects are not taken into ac-count due to lack of knowledge or limitations of computational power [17,1]. Moreover, measurements of model parameters, model validation, or initial and boundary conditions rarely can be achieved with perfect accuracy [16]. Therefore, the model results inevitably contain uncertainties, and one should estimate their magnitudes by applying an uncertainty propagation (UP) method.
Usually a distinction is made between intrusive UP methods, where one substitutes the original model with its stochastic representation, and non-intrusive methods, where the original model is used as a blackbox [19,24]. Intrusive methods are efficient and relatively easy to apply to linear models, e.g. [25]. This, however, represents only a relative small class of models. They can be applied to non-linear models as well, but solution of the resulting equations may become very demanding [27]. Non-intrusive methods can be applied to any type of non-linear model. However, if a single model run requires large execution times, these UP methods may be ineffective, or even computationally intractable.
In this paper, a family of semi-intrusive UP algorithms for multiscale models is introduced. These methods are called semi-intrusive, since they are intrusive only on the level of the multiscale model, that is, in the way the single scale components are coupled together. The single scale components themselves however are treated as black-boxes.
First, the semi-intrusive Monte Carlo (SIMC) method will be introduced, in which the number of samples for the computationally intensive part of the multiscale model (usually microscale dynamics) will be reduced. This leads to a decrease in the computational time for the multiscale UP. A cross validation, which is part of the method, controls the level of subsampling and hence the accuracy of the estimates of uncertainty.
Next, a metamodeling approach is introduced, where a surrogate model substitutes the most expensive single scale model. The metamodel can be constructed by applying, for example, a data-driven approach, like Gaussian process regression [26,18,28], or using a spectral approach, like the stochastic Galerkin method [10]. Since only one component of the multiscale model is approximated by the surrogate, the resulting error in the model output can be small enough to still be able to obtain reliable uncertainty estimates. However, it is expected that this strongly depends on the sensitivity of the output of the multiscale model on that of the single scale component and the method used to build a surrogate of that single scale component.
These UP methods have been tested on two case studies based on reaction-diffusion dynamics with random inputs: a one-dimensional system with slow diffusion and fast reaction and the two-dimensional Gray-Scott model.

Multiscale model
According to the Multiscale Modelling and Simulation Framework (MMSF) [6,3,4,2], multiscale models can be seen as collections of single scale components coupled through the spatio-temporal scales using scale bridging methods. In the current work, this representation of multiscale models is followed.
In Figure 1, an example of a multiscale model with two scales coupled via a scale bridging method is shown. The macro and micro models are denoted by the letters M and µ, respectively. The horizontal arrows are the model initialisation (left) and the final output (right), and the vertical arrows indicate the time execution loop, where at every iteration of the macroscale model the microscale model is executed until completion.
In this work, one class of multiscale models is considered, where the macro and micro models have different time scales [5]: ∆t M and ∆t µ are the time steps at macro and micro levels, and n µ is the number of timesteps at micro level with time step ∆t µ , for each macro time step ∆t M . In this case, the models M and µ can have overlapping or well separated spatial scales. In the case studies presented in section with examples, the case where ∆t M = n µ ∆t µ is considered, meaning that the micro and macro scale are touching each other on the scale separation map [15].
To initialise the multiscale model, the values of the model input parameters ξ should be specified, where ξ ∈ R n is a n-dimensional vector. At each macroscale simulation time point t M ∈ [0, t end ], the macro model calls the micro model, sending the initial state u t 0 , if t M = 0, or the result from the previous time step u t M −∆t M , if t M > 0, with the Quantity of Interest (QoI) u t ∈ R m for t ∈ [0, t end ]. Next, the micro model is run with the time step ∆t µ until it reaches an equilibrium and it produces an output v tµ with v t ∈ R k for t ∈ [0, t end ]. Then, it sends this result back to the macro model, which produces an output u t M , and, then, the simulation time is increased by ∆t M . This process continues until the final simulation time t end is reached. Normally the microscale simulation is some fully resolved model that requires substantial computational resources. This microscale model is called over and over again at every time step of the macroscale model, rendering the microscale computations usually the most expensive part of a multiscale simulation. Usually precise values of the inputs ξ cannot be obtained. Hence, the model output u t M contains an inherent uncertainty. Our goal is to estimate the output uncertainty accurately and in a minimal execution time. Since frequently an execution of multiscale models takes vast amount of time [1], straightforward black-box Monte Carlo methods can be prohibitive. Therefore, in the next section, a family of algorithms, which can perform multiscale UP in a more efficient way, will be introduced.

Multiscale uncertainty propagation
As introduced above, the uncertain inputs are denoted by the vector ξ, and the output of interest is the response of the macro model u t M . The mean value (E u t M ) and the standard deviation (σ u t M ) need to be estimated as measures of uncertainty, assuming that the probability density p u t is unimodal. First, the estimation of the moments by a blackbox Monte Carlo method is shortly described. Then, the semi-intrusive approach is introduced, and more specifically a semi-intrusive Monte Carlo method, and a metamodeling approach will be further explored.

Plain Monte Carlo
An example of uncertainty estimation in the response of a multiscale model by a black-box Monte Carlo (MC) is shown in Fig. 2. We generate N samples of uncertain inputs ξ according to their probability distribution functions p ξ , and run the model N times with these inputs values. The model output is collected, and the mth moment of this output at the simulation time t M is estimated as where u t M j is the value of the macro model output when the model inputs have the values ξ j . The quality of the estimates for the mean value and standard deviation by the MC method is usually provided by confidence intervals, which can be estimated by bootstrap [8].

Semi-intrusive methods
The semi-intrusive methods for multiscale UP are a family of algorithms, which employ the structure of the multiscale models in order to perform an efficient UP, that is, estimating the uncertainties with the comparable quality as the black box MC method, but with a substantially reduced execution time. According to the MMSF, instead of considering the whole multiscale model as a black-box, the model can be seen as a collection of coupled single scale black-box systems. Thus, the semi-intrusiveness of the methods boils down to a limited inspection of the multiscale model, which is only up to the level of single scale components and their coupling.

Semi-intrusive Monte Carlo
The semi-intrusive Monte Carlo (SIMC) is a Monte Carlo method with a reduced number of samples of the expensive component of the multiscale model. The remaining samples are obtained by interpolation. Additionally, a cross-validation is applied to test whether the approximation of the results does not lead to a large error in the estimates of uncertainty.
To perform the method the order of the time execution and the MC sampling is changed as in Fig. 3. At each simulation time t M the macro model produces a sample of size N of the QoI to the micro model, which in turn produces the set of outputs {v an interpolation scheme is used. In this way, the expensive micro model is executed only N µ times for N µ N . The moments of the QoI are approximated by whereũ t M j is the result of the macro model with input of the micro model as a result of the interpolation.
Usually the interpolation method produces results which are not exact to the micro model response, and an error in the uncertainty estimates arises. Thus, a cross-validation on the sample of size N µ must be performed, in order to estimate the effect of the micro model approximation. This test allows to decide whether it is safe to apply the SIMC method, or if instead the MC method with N µ samples should be used.

Interpolation test
In the process of the cross-validation of the results of interpolation the goal is to obtain confidence that the estimation of uncertainty by the SIMC method is close to the uncertainty of the original model response. In other words, the errors must be small. To approximate these errors, their upper bounds will be estimated: This allows to study the random variable u t M −ũ t M , which can be estimated using the N µ samples. Denote by f 1,..,n−1 (ξ n ) the interpolation function, which approximates the micro model output corresponding to the one with the model input values ξ n using the set of original micro model outputs . Call the set of the interpolation results of the micro model output ṽ Then, the macro model is run with these interpolated results for the micro model, resulting in the set . Using this set and the set of the original , the mean and the standard deviation of u t M −ũ t M from inequalities 4 can be approximated: The idea of the interpolation test is to compare for each of the estimators the confidence interval of the MC result with N µ samples and the error from Eq. 6 plus its confidence interval. In Figure 4, an example, when the first is larger than second, is shown, and, in these cases, the results of the SIMC are accepted. Otherwise, another interpolation method can be tested, or the MC estimates from the N µ samples are used. Figure 4: Comparison of the confidence in the estimators obtained by the SIMC and the MC with N µ samples. Given a confidence level, the error of the SIMC method will not be larger than its approximation by Eq. 6 plus its confidence interval. In the illustrated case, when the upper endpoint of confidence interval of the SIMC error is lower than confidence interval of the MC estimates, the SIMC method is applied. Otherwise, the moments are estimated by the MC method.

Metamodeling of a single scale model
Surrogate modeling is a common approach to perform an efficient UP for computationally intensive systems at a reduced amount of time. The idea of these methods is to substitute the original system by its surrogate, which produces a similar output, but their computational time is lower. In the semi-intrusive multiscale metamodeling method, these techniques are applied to a single scale component, which takes the largest portion of the computational time [20]. In this way, the error introduced by the approximation is expected to be small when estimating the uncertainties of the multiscale model.
In Figure 5, an example, where the micro model is substituted by a surrogateμ, is shown. The rest of the multiscale model has the original form. However, since the micro model produces an approximate resultṽ tµ j , the output of the macro model (ũ t M j ) is not the same as with the original model as well.
In this method, the error will always depend on the details of the model. It depends on the prop- erties of the micro model, for example, smoothness, which determines how difficult it will be to approximate the original single scale model. Additionally, the error in the estimates of uncertainty also depends on how sensitive the result of the macro model is to the output of the micro model which is replaced by a surrogate. If, for instance, this sensitivity is low, it is reasonable to expect that the error introduced by the approximation is small. Of course, the error also depends on the method with which the surrogate is build. Next, several ways to obtain a metamodelμ of a single scale component are discussed.
A simplified physical metamodel is one of the options, where one seeks an approximate, maybe lower dimensional solver of the original problem, which would produce the result in a lower amount of time. For instance, this can be done by simplifying the physical description of the modelled process, or by solving the problem on a coarser computational mesh. The error produced due the approximation is problem specific, and one should perform analysis of these errors to preserve approximately correct estimates of uncertainties.
The intrusive Polynomial Chaos (PC) is another method to build a surrogate. The approach is based on the analysis of the solver of the expensive single scale model, and the substitution by its stochastic representation [7,25]. In general, this method works well and is easy to apply to linear and some nonlinear problems. However, the solution can diverge when the method is applied, for example, to problems with a phase transitions [21]. In such cases, one should perform a convergence analysis of the solution in order to obtain correct results of uncertainty estimation [27].
An inspection of the model solver is not required when a data-driven approach is applied. These methods are based on sampling the model output, and then on applying some regression method, such as for example the Gaussian processes [11,26], to obtain the model results in the rest of the function space. The regression methods have some limitations as well. For example, the Gaussian process regression works well only for smooth functions. However, one of the advantages of this method is that it provides an estimate on the precision of the prediction in the interpolated points, hence this allows to control the error of the approximated results.

Examples
In this section, the results of uncertainty estimation for two systems based on reaction-diffusion dynamics are presented. The reaction-diffusion parameters are chosen such that characteristic times for the reactions are much short than for diffusion, rendering this a time-scale separated multiscale system. First, results obtained by the Monte Carlo (MC) method are presented, which are used as a reference solution. Next, the results obtained by a number of variants of the semi-intrusive methods are shown. The semi-intrusive multiscale UP methods that have been tested are SIMC with cubic interpolation, metamodeling with a data-driven surrogate build using the Gaussian Processes (GP) regression, and with a metamodel obtained by the intrusive Polynomial Chaos (PC), which is coupled to the non-intrusive PC. Additionally, results obtained by an intrusive Galerkin method are shown. In this way, the semiintrusive methods are compared with both intrusive and non-intrusive techniques.
The mean value and the standard deviation of the concentration fields in the reaction diffusion systems were measured by these methods as uncertainty estimates. In the first case study, the MC results were obtained using N = 5000 samples, which results in 95% confidence interval for the standard deviation of at most 1.7% of the estimator. In the second case study, the MC sample size is N = 30000, which produces 95% confidence interval for the standard deviation of at most 8% of the estimator. Since the models outputs have nonnormal distributions, these confidence intervals were computed by bootstrap. For the rest of the methods, the parameters are indicated together with the presented results.

Case study 1
The first case study is a 1D reaction-diffusion model with slow diffusion and fast reaction: for x ∈ [0, 1] and t ∈ [0, t end ], with u(x, t = 0, ξ 1 , ξ 2 ) = sin(π(4x − 0.5)) + 1, where d(ξ 1 ) and k(ξ 2 ) are dimensionless diffusion and reaction coefficients with 10% uncertainty. Uncertainty was estimated for results from model simulation with n µ = ∆t M ∆tµ = 100 and n µ = 1000. The mean value of the diffusion coefficient is E[d(ξ 1 )] = 4.05 · 10 −1 , and the mean value of the uncertain microscale coefficient was set by E[k(ξ 2 )] = nµE[d(ξ 1 )] ∆x 2 . In the two experiments with different values of n µ the space step is ∆x = 10 −2 . The estimated mean and standard deviation of the response at different simulation time t (y-axis) by the MC method is shown in Fig. 6.   Fig. 7. The methods are indicated on the x-axis, and the computational time in which the method produced the results is shown on the y-axis, which has a logarithmic scale. The semi-intrusive methods result in a speed up in comparison with the MC method, however, their time is still not so low as the computational time of the intrusive Galerkin method. Above each method bar the mean relative error in the estimate of the standard deviation from each of the methods relative to the MC method is indicated. The MC results serve as a reference solution (r.s.). In the results for both test systems the error does not exceed 0.2%.
In Table 1, the computational times of the methods execution, portions of time spent on micro and macro components, and the speed up by the method versus the MC are included. The left part of the table contains data from the experiment with n µ = 100, and the right part is with data for n µ = 1000. In both cases, the MC uses most of the execution time on the micro model. The SIMC reduces this portion of time to about 70% in the first case, and around 80% in the second case. This leads that uncertainty for the first example were computed almost 9 times faster, and for the second almost 47 times faster than the MC method. This drastic reduction of computational time is because the interpolation takes a constant amount of time regardless the cost of the micro model execution. Therefore, if in the first example the interpolation time was relatively large, in the second example, this number is of little significance. Similarly, the metamodeling with GP and the coupled intrusive and non-intrusive PC compute about 15 times and 5 times faster, respectively, than the MC in the first example, and about 34 and 28 times faster in the second. Therefore, as higher the portion of the execution time of the micro model, as more efficiency is gained by the semi-intrusive methods. The Galerkin method is a highly efficient approach for this system, where it produces the result from 205 to 283 times faster than the MC.
In Figure 8, an analysis of the error in the estimates of uncertainty at the final time step by the SIMC method is presented for the system with n µ = 1000 (the system with n µ = 100 shows a similar result). The upper plots show the estimates of the mean value (left) and standard deviation (right) by the MC and the SIMC methods, which show a good match for both estimates. Additionally, it includes the error estimates of the SIMC results, and, indeed, the MC estimates are within these bounds. The bottom plots show the errors in the estimates of uncertainty from the SIMC method and the MC method with N µ samples in comparison with the MC method with N samples. For both the mean and standard deviation, the SIMC results show lower error than the MC with N µ samples. Indeed, in these examples, the interpolation test is passed.
In this first case study, it is demonstrated that the proposed semi-intrusive methods are more efficient when applied to a multiscale model with a computationally expensive single scale component in comparison with the rest of the system. In the second example, the accuracy of the semi-intrusive UP methods is studied, where the methods are applied to a Gray-Scott model that has a highly non-trivial response.

Case study 2
In this example, a two-dimensional Gray-Scott model [22] is studied: for space variables x, y ⊆ [0, 2.5] 2 , where D u and D v are dimensionless diffusion coefficients, F (ξ 1 ) is a dimensionless feed rate, and k(ξ 2 ) is the dimensionless rate constant of the second reaction. The reaction and diffusion processes are decoupled with n µ = 3 with reaction faster than diffusion. The system has Neumann boundary conditions and initial conditions as follows [22] v(t = 0, x, y, ξ) =      The system is interesting to study, because the model output is very sensitive to the reaction coefficients F (ξ 1 ) and k(ξ 2 ). The model demonstrates a complex pattern formation with a transition map studied in [13]. In Figures 9, examples of two model outputs at the final simulation time with different sets of values of F (ξ 1 ) and k(ξ 2 ) are shown. The model uncertain parameters are F (ξ 1 ) and k(ξ 2 ) with a uniform distribution and an 1% variability range, and D u and D v are constants such that The UP result obtained with the MC of the final time step is presented in Fig. 10. The results of the mean value are still quite close to the patterns from Fig. 9, and this results for u and v are approximately . At the same time, the standard deviations of u and v have a similar pattern. However, since the maximum value in space of u is much greater than v, the relative uncertainty of v represented by the coefficient of variation reaches 100% at some locations, where this value for u is about 36%.
The comparison of the computational time and the error in the standard deviation by the UP methods are presented in Fig. 11, where the MC result is used as a reference solution. The SIMC and metamodeling with GP result in a significant drop in the execution time, and, in contrary, the coupled intrusive and non-intrusive PC and the Galerkin methods are more computationally expensive than the MC method. Moreover, the error in the results of the last two methods exceeds 60%. The high value of the error is due to nonlinear nature of the model, which cannot be approximated by a series of low order polynomials (in our case, the maximum order is N P C = 5). The results obtained by the SIMC and the metamodeling with the GP are much closer to the MC results. In this example, the interpolation test in the SIMC is not passed, and, therefore, the N µ = 50 samples are used to compute uncertainty with the MC, which produces 7.7% error instead of 11% when the result of the SIMC is accepted. Additionally, a more detailed error analysis of the SIMC method is given below.
In Table 2, the total execution time of the UP methods and the portions of time taken by the micro and macro model executions are shown. The increase of the truncation degree in the PC and the Galerkin methods to decrease error makes the methods highly computationally expensive. In the SIMC and the metamodeling with GP, the portion of time spent on micro model is approximately the same, since, first, the system is not strictly multiscale, and, second, un-certainty was computed only for the last time step. Nevertheless, these two methods result in a significant reduction of the computational time, i.e. 216 and 248 times faster than the MC method, respectively.
In Figure 12, the results of the MC and SIMC methods are compared (top row), and the SIMC error is explored (second row) in the mean (left column) and the standard deviation (right column) for the concentration v at the final time step and at y = 0.625. The top left plot illustrates that the MC and SIMC estimates match well. However, the bottom left figure indicates that the error in the results of the M C with N µ samples is smaller that in the SIMC results at some locations. This explains why the interpolation test for the error in the mean value was not passed. The top right plot shows that the match with the MC results for the estimates of the standard deviation is worse, however, indeed the error is bounded by the standard deviation of the absolute difference (dotted line). Moreover, in most of the locations the error in the results of the M C with N µ samples is smaller than in the estimates by the SIMC. According to the interpolation test, this MC result is accepted, therefore, the mean relative error in the standard deviation is 7.7% instead of 11%. This shows that the interpolation test works.

Conclusions
In this work, semi-intrusive multiscale strategies to perform an efficient uncertainty propagation (UP) for multiscale models are proposed and tested on two benchmark problems. It is shown that defining multiscale models according to MMSF, can help to reduce the computational time of the uncertainty study. The number of samples for the expensive single scale model can be reduced. This approach is called the semi-intrusive Monte Carlo (SIMC). Since one of the steps in this method is an interpolation test, the accuracy of estimates is controlled. Likewise, a metamodel of the microscale model can be built before performing UP, allowing to compute an approximate model response without running the expensive single scale model itself. Such surrogate model can be obtained by using a data-driven approach, or by substituting the original microscale model by its computationally cheap version. These methods can be applied to complex or unknown structure of single scale models. However, when one of the single scale models can be rewritten by its stochastic representation, the intrusive Galerkin method can be applied to this single scale, and the non-intrusive PC is applied to other components of the multiscale model. In the first case study, the semi-intrusive approaches were tested on a one-dimensional reactiondiffusion model. It is shown that these methods help to reduce the computational time of UP while only inducing a small error in the uncertainty estimates. Moreover, two examples with different value of the number of micro time steps for each macro step (n µ ) are studied. It is observed that for the larger value of n µ the efficiency (in comparison to the non-intrusive MC) of the semi-intrusive methods is higher than for the smaller n µ . Thus, the strength of these meth- ods is more visible, when the cost of the microscale model is much higher than the cost of the macroscale model, since the time of interpolation or to build a metamodel does not depend on n µ .
In the second case study, the semi-intrusive UP techniques are applied to a Gray-Scott model in order to test if the algorithms are effective and accurate for complex non-linear systems as well. It was observed that the coupled Galerkin method with nonintrusive Polynomial Chaos (PC) and the intrusive Galerkin methods converge very slowly and require a high degree of the truncated PC expansion. This makes the methods non-efficient in terms of computational time. The metamodeling approach with Gaussian Process regression instead decreased the computational time drastically, however, contained about 14% error. The SIMC method produced a relatively small error (7.7%), and significantly reduced the computational cost in comparison to the MC approach. Moreover, the reference solution usually is not available for real world problems, therefore, the magnitude of the estimates error can not be measured. The interpolation test tells whether the micro model can be approximated correctly with N µ samples, and, the SIMC estimate accurately uncertainty using this interpolation function.
In this work, one generic case of multiscale models [5] is considered, where the macro and microscale models have different time scales. However, the semiintrusive multiscale methods can also be applied to other types of the multiscale models, including cases with spatial scale separation. In such cases one can expect that the computational needs for the microscale models can be much more substantial than the benchmarks in this paper, stipulating the absolute need of semi-intrusive methods to make multiscale UP a tractable problem.