E [ G $_{P}$f G $_{P}$g ] = P fg − P fP g , then ˜ G P has almost surely uniformly continuous sample paths. In particular, [13, Example 1.5.10] shows that ˜ G P has almost surely uniformly continuous sample paths with respect to the standard deviation semimetric, and Section 2.1 in that same reference shows that, for bounded F , one can replace this semimetric by that on L $^{2}$( P ). ✷ In the remainder of this section, we give a taste of the broad applicability of this result via three examples. The first example is a simple extension of the framework of Section 2 that allows for more than two possible actions. The second example substitutes the median reward for the mean reward [for a similar setting, see 27]. The third example focuses again on the mean reward, but considers a non-i.i.d., contextual-bandit-type setting in which context- specific actions may be informed by earlier observations [8]. In this example, there are card( A ) ∈ [2 , ∞ ) (finitely many) candidate actions to under- take (think of a discretized dose in the personalized medicine framework). Without loss of generality, we assume that A ⊂ [0 , 1]. We assume that under the distribution P of O ≡ ( X, A, Y ), the reward Y is uniformly bounded and there exists some δ > 0 so that min$_{a}$$_{∈A}$ P ( A = a | X ) ≥ δ with probability one. Here too, the value of a policy π ∈ Π is given by (1), the (within class Π) optimal value is V ⋆ ≡ sup$_{π}$$_{∈}$$_{Π}$ V ( π ) and the regret of π ∈ Π is defined as in (2). Finally, we observe O$_{1}$ ≡ ( X$_{1}$, A$_{1}$, Y$_{1}$ ) , . . . , O$_{n}$ ≡ ( X$_{n}$, A$_{n}$, Y$_{n}$ ) drawn i.i.d. from P . For each π ∈ Π and o ∈ O , let f$_{π}$ ( o ) be given by (3). Let ˜ G n ≡ n 1 / $^{2}$( P$_{n}$ − P ) ∈ ℓ $^{∞}$( F ) be the empirical process on F = { f$_{π}$ : π ∈ Π } . Note the parallel between the LHS of (11) with r$_{n}$ ≡ n 1 / 2 and n 1 / $^{2}$Rem$_{n}$ from (5). We prove the next lemma in Section B.2: Lemma 6. If Π is Donsker, then (8) , (9) , (10) and (12) are met. 14 4.2 Example 1: Maximizing the Mean Reward of a Discrete Ac- tion