{'fraction_non_alphanumeric': 0.04036633942853558, 'fraction_numerical': 0.009723049901201267, 'mean_word_length': 4.815064745577239, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Asynchronous parareal time discretization for partial differential equations

Asynchronous iterations are more and more investigated for both scaling and fault-resilience purpose on high performance computing platforms. While so far, they have been exclusively applied within space domain decomposition frameworks, this paper advocates a novel application direction targeting time-decomposed time-parallel approaches. Specifically, an asynchronous iterative model is derived from the Parareal scheme, for which convergence and speedup analysis are then conducted. It turned out that Parareal and async-Parareal feature very close convergence conditions, asymptotically equivalent, including the finite-time termination property. Based on a computational cost model aware of unsteady communication delays, our speedup analysis shows the potential performance gain from asynchronous iterations, which is confirmed by some experimental case of heat evolution on a homogeneous supercomputer. This primary work clearly suggests possible further benefits from asynchronous iterations.


Introduction
While high-performance computing (HPC) architectures are more and more parallel, the development of efficient and robust algorithms that can fully exploit high degrees of parallelism is still an actual challenge in many scientific computing fields. For solving partial differential equations (PDEs) defined on the interior of arbitrary geometrical shapes, strong advances resulted in highly scalable domain decomposition methods (DDMs) (see, e.g., [24,38]), which feature high rates of speedup and efficiency. Further, although it first appears unnatural to break the time line causality in time-dependent differential equations, thorough applications of DDMs still led researchers to time-parallel methods for allowing actual parallel time-integration iterations. Waveform relaxation methods [21], for instance, solve space-time PDEs by means of independent time-integration processes performed on subregions of the spatial domain, while ensuring global consistency through time-dependent interface conditions. Multiple shooting [9] and time-multigrid [20] methods led to the more recent Parareal time discretization scheme [23], and a general analysis of parallel implicit time-integration algorithms [14], where the time domain itself is decomposed. Basically, two levels of time discretization grid are considered. An approximated solution of the PDE on a coarse grid provides initial values on each time subinterval, when more accurate solutions are thus independently computed on the basis of a sufficiently fine time grid. These fine solutions are then used to enhance new another coarse solution. In this sense, the Parareal computational model can also be considered as an iterative predictor-corrector scheme. We refer to [18] for an introductory insight into time-decomposed time-integration approaches.
Despite such great advances with DDMs, there remain scaling limits upon these parallel algorithms, due to the fact that they require, at some point, sequential global iterations, even if each of these iterations is performed in a parallel fashion. This sequence requirement implies iterative global synchronization, which causes both bottleneck and fault-resilience issues at HPC scales. For such methods then, the only way to completely break sequential computation is to resort to asynchronous iterations [7], which therefore constitute today one of the most interesting algorithmic approaches toward HPC applications. Arisen from linear relaxation schemes (see [12]), asynchronous iterations are basically described by a non-deterministic computational model, such that there is no more a single sequence of parallel iterations for approximating a global solution. Instead, one can independently run a proper sequence of iterations on each subregion of a domain, therefore communication can be completely overlapped by meaningful computation. So far, only fixed-point iterative models have been proved to be asynchronously convergent, under various general contraction conditions (see, e.g., [31,4,13,5]). Still, despite the lower convergence rate of asynchronous iterations, their practical efficiency in execution time over classical fixed-point methods is undoubtedly established, thanks to extensive studies addressing various computational problems (see, e.g., [22,19,8,10,11]). The asynchronous iterations theory has been largely applied within the framework of overlapping Schwarz domain decomposition methods (see, e.g., [15,36,37]), according to their tight relation with block-Jacobi and Gauss-Seidel relaxation schemes (see, e.g., [17]). This included waveform relaxation methods as well (see, e.g., [2,16]), which constitutes so far the only application targeting time-parallel methods. Very recently, non-overlapping Schwarz methods have been investigated (see [29]), just as some spatial domain substructuring approach (see [30]). The goal of this paper is to introduce a novel asynchronous iterations application direction by addressing time-decomposed timeparallel algorithms through the well-known Parareal iterative scheme. Besides a successful experimental investigation in [28], where we only focused on implementation aspects, we present here a theoretical analysis to fully assess both convergence conditions and speedup potential.
Section 2 provides a short general introduction to asynchronous iterations, with most important analysis tools for assessing their application to the Parareal iterative scheme. In section 3 then, we deal with its asynchronous convergence analysis, after deriving the corresponding iterative model. Speedup analysis is conducted in section 4, where we first propose a computational cost model not neglecting communication effects. Section 5 finally presents practical performance experimented on a supercomputer, for a heat evolution problem, and our conclusions follow in section 6.

Computational model
Asynchronous iterations arose in the framework of linear relaxation methods, from the successive experimental and theoretical works of Rosenfeld [33] and Chazan & Miranker [12], respectively. Given a linear system where x is a vector of n unknowns, let M be a nonsingular matrix and consider the mapping with I denoting the identity matrix. Given then that this linear system can be seen as a fixed-point problem, as we have For instance, the Jacobi relaxation scheme consists of taking M as the diagonal part of A. A classical iterative fixed point search generates a sequence {x k } k∈N of vectors such that and such that, for any given initial vector x 0 , with x * being the unique fixed point of f . In a parallel computation context with p processors, p ≤ n, one may assume a decomposition of the form which gives an equivalent sequence of parallel iterations The first step toward asynchronous iterations is the introduction of the free steering (see, e.g., [34]), where, at each iteration, one is allowed to relax an arbitrary subset of unknowns. Note however that parallel computing were not yet targeted at this time, and that, except for free steering, the choice of the unknown(s) to be relaxed was based on deterministic criteria (like Gauss-Seidel relaxation, for instance, which follows a cyclic pattern). Let us then consider, at each iteration, an arbitrary subset defining the relaxed subvectors of unknowns. The corresponding free-steering iterative model is given by where iterations are only implicit. Then, while considering actual parallel free steering, one would be interested in letting each subvector be continuously relaxed at a random pace. In this case therefore, there is no more explicit global iterations, and the sequence {P (k) } k∈N instead models implicit global iterations by accounting each concurrent update of at least one subvector. In such a parallel context, Chazan & Miranker [12] took into account the delay of accessing a subvector, in the sense that this subvector could be relaxed in the meantime, and therefore could become outdated before being used. Consequently, a subvector x k+1 i , i ∈ {1, . . . , p}, is no more necessarily computed on the basis of subvectors x k j , j ∈ {1, . . . , p}, but on the basis of possibly outdated subvectors x This yielded asynchronous iterations Two straight assumptions are made to allow for convergence. First, no component i ∈ {1, . . . , p} should definitively stop being updated, which implies that i belongs to successive subsets P (k) infinitely often, i.e.
∀i ∈ {1, . . . , p}, card{k ∈ N | i ∈ P (k) } = +∞, where card denotes the cardinality. Secondly, delays to access updated components are bounded, so that ∀i, j ∈ {1, . . . , p}, lim Note finally that classical iterations are thus specialization of asynchronous iterations where More generally now, we can consider an arbitrary, non necessarily linear, mapping with E m being the Cartesian product of m sets E = E 1 ×· · ·×E p , and a fixed-point problem The study of such a generalization goes back to Baudet [4] for addressing Newton asynchronous iterations. This generalizes the model (2) to in the sense that (2) corresponds to the case m = 1.

Convergence conditions
It is well known (see, e.g., Theorem 2.1 in [3]) that the sequential iterative model (1) is convergent if and only if where ρ(.) denotes the spectral radius. Chazan & Miranker [12] showed that the asynchronous iterative model (2) is convergent if and only if where |.| denotes the absolute value, which is slightly more restrictive, since we always have (see, e.g., Corollary 6.3 in [6]) Still, to study the application of asynchronous iterations to the Parareal scheme, we would rather need the framework of arbitrary mappings, for which we recall here two most general sufficient contraction conditions guaranteeing asynchronous convergence. Let each set E i , with i ∈ {1, . . . , p}, be a Banach space provided with some norm . (i) , and consider weighted maximum norms where w is a positive vector with real entries w i , i ∈ {1, . . . , p}. The following result goes back to El Tarazi [13]: 1982). The asynchronous iterative model (5) is convergent if there exists a positive vector w and a positive real α < 1 such that (1) , . . . , x (m) ), Y = (y (1) , . . . , y (m) ).
Bertsekas [5] then provided a more general contraction framework based on the identification of an implicit sequence of nested sets which converges toward the solution set {x * } (actually, this even applies for mappings admitting several fixed points): Theorem 2 (Bertsekas, 1983). The asynchronous iterative model (5), with As mentioned above, the sets sequence {S (l) } l∈N is only implicitly generated all along the successive updates of the components x i , and, for more clarity, we do use another sequence variable, l, since several asynchronous iterations k could be performed within a same set S (l) before entering the next one, S (l+1) . The main difficulty in applying such a result to a specific mapping f is to be able to identify the sets S (l) and S (l) i . Bertsekas & Tsitsiklis [7] showed that this was more general than the contraction framework based on weighted maximum norms. When, for instance, Theorem 1 is applicable, with m = 1, then we have (see [6, section 6.3] and [7]) and S (l)

Computational model
Let us consider a time-dependent PDE, with some given boundary condition, and initial values, Let the time domain [0, T ] be given by and assume initial values . . , p}. Independent subproblems can then be formulated by which is equivalent to the global problem (8) if and only if Now let, for i ∈ {1, . . . , p}, and let F be a function which, to initial values λ i−1 (s), associates a sufficiently accurate solution of the subproblem (9), at time T i , i.e., Similarly, let G be a function which, to initial values λ i−1 (s), associates a cheap approximation of the solution of the subproblem (9), at time T i , i.e.,

From initial guess
. . , p}, the Parareal iterative scheme [23,14] which should converge to a collection {λ * i } satisfying the interface conditions (10). This algorithm is thus expected to provide the solution u (T ) of (8) at time T , without applying the whole sequence To easily assess such a computational model within the previous framework of general asynchronous iterations, let us consider linear operators and a vector λ = (λ 0 , λ 1 , . . . , λ p ).
Then, (11) simplifies to . . , p}, which corresponds to synchronous relaxations (1) of a problem recalling that the iterations mapping f is given by Remark. This corresponds to a preconditioned Richardson iterative scheme, as also formulated, e.g., in [32].
Nonetheless, a straightforward application of asynchronous relaxations (2) would yield a computation of the form which is not of great interest, due to the synchronization on G(λ k+1 i−1 ). We would therefore rather be interested in seeing what happens with a computation, at least, of the form which corresponds to the more general asynchronous iterations (5), with m = 2, and where the iterations mapping f is given by It is easy to verify that which ensures an equivalent fixed-point problem. We recall that the asynchronous iterations variable k grows faster than the synchronous iterations one, therefore, the delays expressed in (12) do not have to be seen as implying a retarded iterative procedure. Note, finally, that the mapping f is explicitly expressed without linearity assumption on F and G. In the following convergence analysis of the Parareal asynchronous iterations (12), such an assumption will be sometimes made only to keep the development simple and to use same notations as with the linear synchronous formulation. Furthermore, to fully exploit the domain decomposition point of view, one will notice that, in the asynchronous framework, the extension of these results to a general formulation using F i and G i is quite straightforward.

Convergence conditions
Let . ∞ denote a maximum norm (6) where w is a vector of 1. We first restate preliminary results in our linear synchronous framework.
and . denoting any given norm.
Proof. See Theorem 4.5 in [18] for a possible proof, or directly consider the fact that linear fixed-point formulation implies Remark.
One is therefore interested by the case θ < 1 which ensures a stable iterative procedure.
Proof. It directly follows from Theorem 3 that convergence is guaranteed if If then G < 1, we can take θ = G , which leads to the result.
Remark. The Parareal synchronous iterative model (11) is thus asymptotically convergent We need here some words about the terminology. It is well established (see [14]) that Parareal iterations terminate in finite time k = p, due to the sequential propagation of the accurate solution, from [T 0 , T 1 ] to [T p−1 , T p ]. By "convergence", we therefore mean the fact that the iterative procedure actually allows one to reach a given precision by decreasing the maximum error over the whole time domain [T 0 , T p ], which thus suggests the possibility to converge before getting sequential. To be rigorous though, "convergence", here, is a shortcut word for "convergence with respect to the maximum norm".
In the following, we would now like to extend such results to the asynchronous framework.
With such an error bound, a convergence condition trivially follows.

Corollary 2.
The Parareal asynchronous iterative model (12) is convergent if Proof. This is straightforward, from Theorem 4.
Remark. The proof of Theorem 4 exhibits which clearly indicates that Parareal asynchronous iterations (12) match the contraction framework of Theorem 1, which also implies convergence under the same sufficient condition (16).
One of the advantages of asynchronous iterations is that the algorithm can be studied on the basis of only local behavior f i , independently of its input which, therefore, can be provided by whatever other local contracting mapping f i−1 . This is an interesting feature which suggests much more flexibility for further developments. Indeed, for instance, in the proof of Theorem 4, one can notice that the whole analysis is based on only Therefore, to fully account for the domain decomposition context, the resulting error bound can be directly restated using G i and F i , with Furthermore, without linearity assumption on functions G i and F i , the same result can be simply obtained, for by generalizing the arbitrary norm . to a distance function over the space of functions u (t) , with u (t) (s) := u(s, t).
In conclusion, it is interesting to see that, while asynchronous iterations generally require more restrictive convergence conditions, they can be successfully performed in any Parareal-based solver, for large numbers of time subintervals (p → ∞), without verification of additional properties, since convergence conditions are very nearly the same.
When, additionally, it happens that the Parareal iterative scheme itself does not actually converge, a finite-time termination is still ensured, at k = p, with the solution provided by a sequential application of F from [T 0 , T 1 ] to [T p−1 , T p ] (see [14]). We use here the idea of nested sets {S (l) } l∈N from Theorem 2 to show the finite-time termination of Parareal asynchronous iterations. Note that deriving nested sets of the form (7) applies only when the convergence condition is satisfied, what is not assumed here for finite-time termination. We therefore instead define S (l) so as to have the following property.
Proof. Let us define sets so that we have (17) and Now let where f is the Parareal asynchronous iterations mapping (see (13)). Then we have At last, let Γ denotes the set of all functions u (t) , with u (t) (s) := u(s, t), and let us define sets (18) to (20) ensure that Parareal asynchronous iterations (12) generate such a sequence {S (l) } l∈N , which concludes the proof.

Computational cost
Farhat & Chandesris [14] established a bound of 1/2 on the parallel efficiency (speedup per processor) of the Parareal scheme, assuming that at least one corrective iteration would be needed (which implies at least k = 2 iterations). It would therefore be of some interest to see at which extent asynchronous iterations could improve the parallel performance, since it is intuitively expected that, here as well, the finite-time termination causes the possible time saving to be bounded.
Without questioning this efficiency bound, Aubanel [1] provided additional insights by taking into account different possible implementations. We consider here the distributed approach therein proposed, which features better scaling potential. Basically, it consists of distributing the sequential application of G over the set of processors, as described by Algorithm 1. We assume the case where we have p processors, so that λ i , with i ∈ {1, . . . , p}, is held by the processor i. Let then C G and C F denote the computational costs of applying G and F , respectively. The cost of a sequential application of F over the whole time interval is thus given by What is remarkable with such a distributed implementation is that the processor i − 1 starts applying F before the processor i. Therefore, in a perfect load balance context, one may expect λ k+1 i−1 to be available to the processor i at the moment when it finishes applying F , what would result in a computational cost, after k iterations, given by (see [1]) where p C G corresponds to the cost of the initialization phase in Algorithm 1.
While accurate enough within the scope of [1], this ideal case however assumes negligible communication, vector addition and memory access costs, which is questionable in real world situations. Let then, at least, C C,G (i, k) give communication overhead costs which accumulate with C G on the processor i, at the iteration k, without being overlapped by the application of F on the processor p. It yields: k := k + 1 26: until λ k λ * Remark. Actually, not only communication cost is not negligible, but it can even be higher than C G , since both are closely proportional to the size of λ i . Note that C C,G (j, l) is of the form C C,G (j, l) = C C (j, l) + β j,l C G , β j,l ∈ {0, 1}, where C C (j, l) is a non-overlapped communication cost.
Still, let us assume a constant uniform communication cost and, for instance, a four-steps fine integration F =F •F •F •F , withF being some one-step integration which costs nearly as much as G. Then, by introducing non-negligible communication C into Aubanel's model [1], Algorithm 1 would behave like which shows that the overhead would lie only in the initialization or the first iteration phase, implying that Our general cost model including C C,G therefore becomes handy when variations can occur in the communication delays, due for instance to a fault recovery procedure or simple perturbation on the network. We see from the example that a little cost increase in the second message transfer toward the processor 4 would result in an overhead cost. At the same time, the impact of these variations can also be expected to decrease with k. Nevertheless, instead of an exhaustive analysis, we are rather interested in assessing a global potential effect of communication. Let us then consider a rough estimation, C C,G , of the average overhead cost at each iteration, on each processor. This leads to: The computational cost of k distributed Parareal iterations (Algorithm 1) is of the form Proof. From (21), we have The proposition thus follows.
Remark. While k might be expected to remain relatively small, the effects of C C,G however increase with the number p of processors. It is important to note that accounting such overhead costs does not contradict the perfect load balance assumption. In this sense, this would be a minimum relevant ideal model for the distributed Parareal scheme.
Let us finally introduce asynchronous iterations into Algorithm 1, to obtain Algorithm 2. Still assuming perfect load balance, the corresponding computational cost is that of the if i < p then Then we have: The computational cost of k distributed Parareal asynchronous iterations (Algorithm 2) is of the form Proof. This is straightforward, since communication is completely overlapped.

Performance gain
The error bounds from Theorem 3 and Theorem 4 indicate that the convergence speeds of Parareal synchronous and asynchronous iterations depend on how small the respective factors are. As intuitively expected, we verify the following comparison.
Proof. Since G < 1, we have Then we have Consequently, which concludes the proof.
Nonetheless, if we consider synchronous and asynchronous iterations variables k and k, respectively, we should bear in mind that k grows much faster than k. Therefore, we can reasonably expect that σ( k) also grows faster than k, sufficiently to make Parareal asynchronous iterations reach the desired precision sooner than synchronous ones do.
To be more practical, besides convergence rate, we see from Algorithm 2 that, either λ ki+1 i−1 is always an updated available input for the processor i, and hence, this reduces to a classical Parareal execution, or it happens that for some processors i, and hence, one has The advantage of the asynchronous scheme is to account for the fact that this could be enough to reach the desired precision, instead of waiting for an updated input in order to necessarily output an exact Parareal update. Note that, in any case then, we should have where k would be the number of synchronous iterations. From our cost model in Proposition 2 and Proposition 3, we deduce: Corollary 3. The speedup over classical distributed Parareal is bounded as Proof. If Now, considering that where we recall that p C G is the initialization cost. With the maximum gain expected at κ( k) = k, this bound reaches its maximum at k = 1, which leads to the result.
Still, we should point out the fact that, in HPC context, and particularly for time-parallel time-integration (see, e.g., introduction and comments in [14,1,32]), one is generally more interested in being able to improve performance by providing more and more processors, in order to keep a given amount of work per processor (strong scaling). In this sense, the bound of 50% over the traditional parallel efficiency is not actually limiting the desired scaling potential. Let us then highlight, for a more complete performance analysis, that the asymptotic speedup from the distributed Parareal over sequential application of F is given by while that of the distributed async-Parareal reduces to which finally implies that As expected, the performance gain from async-Parareal should increase with C C,G , even in a perfectly well balanced configuration, and we saw that C C,G can be of the same order as C G . It may however seem surprising that the asymptotic speedup does depend on k, and not on κ( k) too. When looking at the classical speedup though (in the proof of Corollary 3), we may expect that, for given C C,G , F and G, the ratio k/κ( k) is bounded by a constant. This is very likely to be true, as the current async-Parareal scheme is also quite close to the classical Parareal one. Further analysis may be needed to fully assess such a relation. Still in this case, while the speedup would decrease with k, the asymptotic speedup rather increases. Indeed, in the distributed Parareal algorithm, the amount of communication decreases with k (as partial termination is taken into account), which lessens the impact of C C,G . However, for very high numbers of processors, this diminution is generally negligible, as p − k remains significantly high for a long period of time.
In the end, while the practical performance gain from applying asynchronous iterations to an existing distributed scheme is no more questionable for spatial parallelism on PDEs, the case of time parallelism is quite less obvious. We therefore present here experimental results only conducted to confirm the possibility of performance gain here as well. Especially, such a configuration where each processor depends on only one other processor is probably the worst for squeezing the most out of asynchronous iterations. Furthermore, to give full advantage to the classical distributed Parareal, we experimented using perfect static load balance, on a homogeneous supercomputer with high-speed communication (which minimizes C C,G ). We reported another, more detailed, experimental investigation in [28], where we mainly focused, throughout the paper, on implementation aspects within the Message Passing Interface (MPI) framework. This issue about asynchronous iterations programming has been more generally discussed in [25,27], where an MPI-based programming library has been proposed, including tools to evaluate the loop stopping criterion of Algorithm 2 (see [26] for a full introduction to exact residual error evaluation under asynchronous iterations).

Problem and experimental settings
The experimental case consisted of simulating an apartment gradually cooled by an air conditioner. We used a simple heat evolution model of the form ∂u ∂t = ∆u, by setting a constant Dirichlet boundary condition on the air conditioner to simulate a continuous cooling at 73.4 degrees Fahrenheit (23 degrees Celsius), from an initial homogeneous temperature at 86.0 degrees Fahrenheit (30 degrees Celsius). Spatial discrete equations were obtained through P 1 -Lagrange finite-elements integration upon a mesh composed of 171,478 tetrahedrons and 33,796 nodes (see Figure 1), generated by the TetGen [35] free software. The computer aided design (CAD) was performed using the CATIA V5 proprietary software tool (©Dassault Systèmes). Operators G and F consisted of either backward Euler or which implied that Each spatial linear system resolution was sequential and based on LU factorization. Figure  2 shows the evolution of the temperature over time. We performed several parallel runs on an SGI Altix ICE supercomputer of 68 nodes interconnected by a QDR Infiniband network (40Gb/s). Each node consisted of two sixcores Intel Xeon X5650 CPUs (12 cores per node) at 2.66GHz, and 21GB RAM allocated for running jobs. The MPI library SGI-MPT was loaded as communication middleware. Execution parameters included the number p of MPI processes, which always equals the number of time intervals, and the loop stopping criterion We refer to [27] for the implementation of such a criterion in case of asynchronous iterations k 1 to k p . Table 1 and Table 2 report average results, where T denotes wall-clock execution times in seconds. We recall that T p is the simulated end time, and that C C,G is the average non-overlapped overhead costs induced by communication at each iteration. This does not directly corresponds to communication costs, and therefore is hardly accurately predictable. We rather deduced values C C,G which make C (k) match the measured execution times. We also measured, on average, C G 0.14 sec.

Performance comparison
Compared to these C C,G values, it seems clear that, even in such an ideal HPC computational environment, the actual impact of communication costs cannot be neglected, despite the optimized distributed implementation from Aubanel [1]. As expected, Table 1 shows that the overhead costs at one iteration tend to diminish as k grows, which makes the average C C,G diminish as well, while the sum of these costs can however keep increasing, which explains the increasing better performance of asynchronous execution. On the other hand, estimated values C ( k) are quite close to measured times (mostly in Table 1). This is explained by the fact that asynchronous execution somehow converts overhead communication costs into additional iterations which are accounted by κ( k). Dealing with the prediction of κ( k) could be a less harassing task, compared to C C,G . One can notice for instance that in Table 1 and Table 2 respectively, which confirms, for these test cases, our expectation of a generally bounded ratio. Finally, with a stopping criterion (22), it happened that async-Parareal generally tended to provide a better precision, according to the sequential solution from F . While this was probably due to the particular termination delay of asynchronous iterations, it did not prevent us, in case of Table 1, from obtaining a faster solver, for p ≥ 32. For instance, around 12 minutes are saved over 44, using 90 processors. On the contrary, Table 2 shows cases where, with sufficiently low values C C,G (regarding numbers of iterations), synchronous iterations keep performing better than asynchronous ones.

Conclusion
Efficiently applying asynchronous iterations in time domain decomposition is rather challenging, and this primary work certainly leaves an amount of further desirable benchmarks and improvements. While we tackled here the basic Parareal scheme, it is somehow important to note that introducing asynchronous iterations is a design approach quite different from, and not concurrent to, classical scheme improvement, as they could actually be applied to more advanced Parareal schemes as well, as long as communication is a possible factor of performance limit in real situations. What is particularly interesting is that convergence conditions (in maximum error norm) barely change, and are even asymptotically the same, while also finite-time termination remains guaranteed. Still, despite the potential theoretical performance gain and some experimental results confirming it in practice, it is clear that such a pipeline parallel configuration drastically limits the benefits from asynchronous iterations, compared to spatial parallelism where each processor generally depends on several others, such as in undirected grid configurations. Hopefully, as pointed out by this first theoretical analysis, much more flexibility is now provided to deal with this issue, which is currently under consideration.