The proof concludes by noting that (12) includes the condition that almost all sample paths of ˜ G P are uniformly continuous on F with respect to ‖·‖ , and thus the right-hand side above is o$_{P}$ (1). In conclusion R $^{(}$̂ π$_{n}$ ) = o$_{P}$ ( r − 1 n ). Proof of Lemma 6. If the bounded class Π is Donsker, then it is totally bounded in L $^{2}$( P ), so (8) is met. By Lemma A.1, Π is then compact. Let θ be a Lipschitz function from [ − 2 , 2] to [0 , 1] such that θ (0) = 1 and θ ( u ) = 0 if | u | ≥ min$_{a}$$_{̸}$$_{=}$$_{a}$ $^{′}$∈A | a − a $^{′}$| . Introducing θ is merely a trick to generalize Lemma A.2. In particular, (A.1) becomes for every policy π : X → A , yielding for every π$_{1}$, π$_{2}$ ∈ Π, hence (9). We also generalize (A.10), which becomes for every π ∈ Π. The above equality and (B.17) imply ‖ f$_{π}$$_{1}$ − f$_{π}$$_{2}$ ‖ ≲ ‖ π$_{1}$ − π$_{2}$ ‖ for every π$_{1}$, π$_{2}$ ∈ Π, hence (10). The second part of the proof of Lemma A.7 can easily be generalized to derive that F is Donsker from (B.18) and the fact that Π is itself Donsker. Therefore, (12) is met and the proof is complete. We start by proving Lemma 9. B.3 Proofs for Section 4.3 B.2 Proof for Section 4.2 Q ( π ( X ) , X ) = ∑ a ∈A θ ( π ( X ) − a ) Q ( a, X ) |V ( π$_{1}$ ) − V ( π$_{2}$ ) | ≲ ‖ π$_{1}$ − π$_{2}$ ‖ (B.17) f$_{π}$ ( O ) = ∑ a ∈A θ ( π ( X ) − a ) ( θ ( A − a ) Y − Q ( A, X ) P ( A | X ) + Q ( a, X ) ) − V ( π ) (B.18) 32