triangle inequality we bound ‖ ˜ h − h$_{0}$ ‖ L $^{2}$( X ) ≤ ‖ ˜ h − h$_{0}$$_{,K}$ ‖ L $^{2}$( X ) + ‖ h$_{0}$$_{,K}$ − h$_{0}$ ‖ L $^{2}$( X $_{)}$. (78) Recall the definition of the empirical projection P$_{K,w,n}$ from expression (15), and observe that ˜ h = P$_{K,w,n}$h$_{0}$ and that P$_{K,w,n}$h = h for all h ∈ B$_{K,w}$ . Also recall the definition of L 2 $_{w,n}$( X ) as the space of functions with finite norm ‖ · ‖ w,n where ‖ f ‖ 2 w,n = 1 n ∑ n i $_{=1}$f ( X$_{i}$ ) $^{2}$w$_{n}$ ( X$_{i}$ ). Since the empirical and theoretical L $^{2}$( X ) norms are equivalent over B$_{K,w}$ wpa1 under the condition ‖ ˜ B ′ w ˜ B$_{w}$/n − I$_{K}$ ‖ = o$_{p}$ (1) (see Section 4.2). Therefore, we have ≍ ‖ P$_{K,w,n}$ ( h$_{0}$ − h$_{0}$$_{,K}$ ) ‖ 2 w,n wpa1 (80) ‖ ˜ h − h$_{0}$$_{,K}$ ‖ 2 L $^{2}$( X ) = ‖ P$_{K,w,n}$ ( h$_{0}$ − h$_{0}$$_{,K}$ ) ‖ 2 L $^{2}$( X ) (79) P$_{K,w,n}$ ( h$_{0}$ h$_{0}$$_{,K}$ ) 2 w,n wpa1 (80) where the second line is by equivalence of the empirical and theoretical L $^{2}$( X ) norms wpa1, and the final line is because P$_{K,w,n}$ is an orthogonal projection on L 2 $_{w,n}$( X ). Finally, Markov’s inequality yields ‖ ( h$_{0}$ − h$_{0}$$_{,K}$ ) ‖ 2 w,n = O$_{p}$ ( ‖ h$_{0}$ − h$_{0}$$_{,K}$ ‖ 2 L $^{2}$( X $_{)}$). Proof of Lemma 3.1. We use a truncation argument together with exponential inequalities for random matrices. Let M$_{n}$ ≍ ( ζ$_{K,n}$λ$_{K,n}$ ) (2+ δ ) /δ (with δ as in Assumption 2(iii) be a sequence of positive numbers and let ̂ Ω$_{1}$ = 1 n ∑ i =1 (Ξ$_{1}$$_{,i}$ − E [Ξ$_{1}$$_{,i}$]) (82) n Ω$_{2}$ = 1 n i =1 (Ξ$_{2}$$_{,i}$ − E [Ξ$_{2}$$_{,i}$]) (83) Ξ$_{1}$$_{,i}$ = ϵ 2 i ˜ b K $_{w}$( X$_{i}$ ) ˜ b K $_{w}$( X$_{i}$ ) $^{′}${‖ ϵ 2 i ˜ b K $_{w}$( X$_{i}$ ) ˜ b K $_{w}$( X$_{i}$ ) $^{′}$‖ ≤ M 2 n } (84) Ξ$_{2}$$_{,i}$ = ϵ 2 i b K $_{w}$( X$_{i}$ ) b K $_{w}$( X$_{i}$ ) ϵ 2 i b K $_{w}$( X$_{i}$ ) b K $_{w}$( X$_{i}$ ) > M 2 n . (85) i b $_{w}$( X$_{i}$ ) b $_{w}$( X$_{i}$ ) $^{′}${‖ ϵ i b $_{w}$( X$_{i}$ ) b $_{w}$( X$_{i}$ ) $^{′}$‖ > M n } . (85) Clearly ̂ Ω − Ω = ̂ Ω$_{1}$ + ̂ Ω$_{2}$, so it is enough to show that ‖ ̂ Ω$_{1}$ ‖ = o$_{p}$ (1) and ‖ ̂ Ω$_{2}$ ‖ = o$_{p}$ (1). Control of ‖ Ω$_{1}$ ‖ : By definition, ‖ Ξ$_{1}$$_{,i}$ ‖ ≤ M 2 $_{n}$. It follows by the triangle inequality and Jensen’s 39 7.2 Proofs for Section 3 ≤ ‖ ( h$_{0}$ − h$_{0}$$_{,K}$ ) ‖ 2 w,n (81)