been randomly drawn from Rademacher laws † with parameters g$_{1}$ (1 , X$_{1}$ ) , . . ., g$_{n}$$_{−}$$_{1}$ (1 , X$_{n}$$_{−}$$_{1}$ ), respectively. Now, let us describe how O$_{n}$ is sampled by specifying how action A$_{n}$ is random- ized, therefore completing the description of the sampling design. Based on O$_{1}$, . . ., O$_{n}$$_{−}$$_{1}$ , we define where the least-square loss function L is given by L ( Q$_{β}$ )( O ) ≡ ( Y − Q$_{β}$ ( A, X )) $^{2}$. This yields the mapping g$_{n}$ : {− 1 , 1 } × X → [ t$_{n}$, 1 − t$_{n}$ ] ⊂ [0 , 1] given by Once X$_{n}$ is observed, we sample A$_{n}$ conditionally on O$_{1}$, . . ., O$_{n}$$_{−}$$_{1}$ and X$_{n}$ from the Rade- macher law with parameter g$_{n}$ (1 , X$_{n}$ ), we then carry out action A$_{n}$ , observe Y$_{n}$ ≡ Z$_{n}$ ( A$_{n}$ ), and form O$_{n}$ ≡ ( X$_{n}$, A$_{n}$, Y$_{n}$ ). Remark 13. For any g : {− 1 , 1 } × X → [0 , 1], let P g be the distribution of O defined by (i) sampling ( X, Z ( − 1) , Z (1)) from P , (ii) sampling A from the Rademacher law with parameter g (1 , X ), (iii) setting O ≡ ( X, A, Z ( A )). Note that W ≡ g ( A, X ) is a deterministic function of g and O . Therefore, we can augment O with W , i.e. , substitute ( O, W ) for Remark 12. The sampling design looks for a trade-off between exploration and exploitation. Here, exploitation consists in using the current best estimates Q$_{β}$$_{n}$ (1 , X$_{n}$ ) and Q$_{β}$$_{n}$ ( − 1 , X$_{n}$ ) of E$_{P}$ [ Z (1) | X = X$_{n}$ ] and E$_{P}$ [ Z ( − 1) | X = X$_{n}$ ] to favor the action which seems to have the larger mean reward in context X$_{n}$ . If the absolute value of the difference between the two estimates is larger than ξ$_{n}$ , then the supposedly superior action is carried out with probability (1 − t$_{n}$ ). Exploration consists in giving the supposedly inferior action a probability t$_{n}$ to be undertaken nonetheless. This allows, for instance, to correct a possibly poor estimation of E$_{P}$ [ Z (1) | X ] and E$_{P}$ [ Z ( − 1) | X ] in some strata of X . ✷ $^{†}$The Rademacher law with parameter p ∈ [0 , 1] is the law of A ∈ {− 1 , 1 } such that A = 1 with probability p . 19 1 − g$_{n}$ ( − 1 , x ) ≡ g$_{n}$ (1 , x ) ≡ G$_{n}$ ( Q$_{β}$$_{n}$ (1 , x ) − Q$_{β}$$_{n}$ ( − 1 , x ) ) . β$_{n}$ ≡ arg min β ∈ B$_{n}$ n − 1 ∑ i =1 L ( Q$_{β}$ )( O$_{i}$ ) g$_{i}$ ( A$_{i}$ | X$_{i}$ )