4.4 Example 3: Sequential Decisions to Maximize the Mean Re- ward of a Binary Action The sampling design. This section builds upon [8]. Let { ( X$_{n}$, Z$_{n}$ ( − 1) , Z$_{n}$ (1) }$_{n}$$_{≥}$$_{1}$ be a sequence of random variables drawn independently from a distribution P such that, if { ˜ A$_{n}$ }$_{n}$$_{≥}$$_{1}$ is a sequence of independent actions drawn from the conditional distribution of A (an action with support {− 1 , 1 } ) given X under the same P as in Section 1.3, then the resulting sequence { ( X$_{n}$, ˜ A$_{n}$, Z$_{n}$ ( ˜ A$_{n}$ )) }$_{n}$$_{≥}$$_{1}$ is an i.i.d. sample from P . For notational simplicity, we assume that all the random variables Z$_{n}$ ( − 1) and Z$_{n}$ (1) take their values in [0 , 1]. In this example, however, the distribution ν$_{n}$ of ( O$_{1}$, . . . , O$_{n}$ ) does not write as a product because the actions A$_{1}$, . . . , A$_{n}$ are not i.i.d. On the contrary, once sufficiently many obser- vations have been accrued to carry out inference then, sequentially, each new randomized action A$_{n}$ is drawn conditionally on X$_{n}$ and an estimate derived from the previous obser- vations O$_{1}$ ≡ ( X$_{1}$, A$_{1}$, Y$_{1}$ ) , . . . , O$_{n}$$_{−}$$_{1}$ ≡ ( X$_{n}$$_{−}$$_{1}$, A$_{n}$$_{−}$$_{1}$, Y$_{n}$$_{−}$$_{1}$ ) yielded by the previous actions, where Y$_{i}$ ≡ Z$_{i}$ ( A$_{i}$ ) for i = 1 , . . . , n − 1. Let us describe how the data-adaptive design unfolds. Let { t$_{n}$ }$_{n}$$_{≥}$$_{1}$ and { ξ$_{n}$ }$_{n}$$_{≥}$$_{1}$ be two nonincreasing sequences with t$_{1}$ ≤ 1 / 2, lim$_{n}$ t$_{n}$ > 0 and lim$_{n}$ ξ$_{n}$ > 0. For each n ≥ 1, let G$_{n}$ be a nondecreasing κ$_{n}$ -Lipschitz functions approximating u ↦→ 1 { u ≥ 0 } in the sense that G$_{n}$ ( u ) = t$_{n}$ for u ≤ − ξ$_{n}$ , G$_{n}$ (0) = 1 / 2, G$_{n}$ ( u ) = 1 − t$_{n}$ for u ≥ ξ$_{n}$ . We also suppose that lim sup$_{n}$ κ$_{n}$ < ∞ . For each n ≥ 1, let Q$_{n}$ ≡ { Q$_{β}$ : β ∈ B$_{n}$ } be a working model consisting of functions mapping {− 1 , 1 } × X to [0 , 1]. Each Q$_{β}$ ∈ Q$_{n}$ can be viewed as a candidate approximation to the function ( A, X ) ↦→ 1 { A = 1 } E$_{P}$ [ Z (1) | X ] + 1 { A = − 1 } E$_{P}$ [ Z ( − 1) | X ]. For some n$_{0}$ ≥ 1, A$_{1}$, . . . , A$_{n}$$_{0}$ are independent draws from the conditional distribution of A given X under P . For every i = 1 , . . . , n$_{0}$ , let g$_{i}$ : {− 1 , 1 } × X → [0 , 1] be defined pointwise by g$_{i}$ ( a, x ) ≡ P ( A = a | X = x ). Set n > n$_{0}$ , suppose that we have fully speci- fied how O$_{1}$, . . . , O$_{n}$$_{−}$$_{1}$ have been sampled through the description of how A$_{1}$, . . . , A$_{n}$$_{−}$$_{1}$ have 18