{'fraction_non_alphanumeric': 0.041794485326713364, 'fraction_numerical': 0.021633123503856724, 'mean_word_length': 4.9109108059951785, 'pattern_counts': {'":': 0, '<': 18, '<?xml version=': 0, '>': 6, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
The role of expecting feedback during decision-making under risk

Sometimes choice is followed by outcome feedback and other times it is not. It remains unknown whether humans prefer gambling when they expect feedback to be revealed. Regarding this question, decision-making theories make alternative predictions. Some theories have proposed that choice is influenced by whether one expects to be disappointed in the future. Given that feedback is sometimes disappointing, these theories predict increased aversion towards gambling when feedback is expected compared to when feedback is not expected. The opposite effect is predicted by theories of curiosity, which postulate reduction of uncertainty as an important behavioural drive. Given that feedback reduces uncertainty, these theories predict that gambling will be favoured when feedback is expected. To examine whether expecting feedback influences gambling behaviour, we recorded functional neuroimaging data while participants performed a novel decision-making task requiring to chose between a sure option and a gamble. Crucially, participants expected to receive feedback in some trials but not in other trials. Consistent with theories of curiosity, we found that expecting feedback increased gambling propensity. At the neural level, at option presentation the increased value of gambling during feedback was reflected in activity in the ventral striatum. This suggests that, together with its established role in signalling reward, the ventral striatum also processes a form of epistemic value. Our study demonstrates that gambling becomes more attractive when feedback is expected and suggests that striatal activity could signal the value of feedback information.


INTRODUCTION
Sometimes choice is followed by outcome feedback and other times it is not. Playing slot machines is an example of the former case, because each individual decision is followed immediately by a clear outcome feedback. Conversely, consider a company engaged in a door-to-door flyer marketing campaign. Although the global effect of the campaign can be estimated, the outcome obtained by each single flyer (e.g., whether it has encouraged people to buy the product or not) will remain unknown. An open question is whether decision-making changes when humans expect to receive feedback after choice compared to when feedback is not expected. Answering this question is important because understanding the role of expecting feedback can arbitrate between influential decision-making theories. It has been proposed that expecting feedback evokes anticipatory feelings of disappointment, and that these influence choice (Bell, 1985;Camille et al., 2004;Coricelli et al., 2005;2007;Loomes & Sugden, 1986). Given that feedback is sometimes disappointing, this proposal implicates an increased avoidance of gambling when feedback is expected. The opposite prediction arises from perspectives emphasizing curiosity as an important behavioural drive (Baldassarre & Mirolli, 2013;Friston et al., 2015;Gottlieb, 2012;Gottlieb et al., 2014;Kidd & Hayden, 2015).
According to this view, feedback is associated with epistemic value because it reduces uncertainty, and thus expecting feedback should enhance gambling.
Motivated by this reasoning, our study examined the cognitive and neural processes underlying the impact of expecting feedback on decision-making under risk. We recorded functional magnetic resonance imaging (fMRI) data while participants performed a new gambling task ( fig. 1A) in which, on each trial, they chose between a sure monetary gain or loss and a 50-50 gamble. The two options always had equivalent expected value (EV; i.e., the sum of possible outcomes multiplied by their probability -here always 0.5 for the gamble). Across trials, we manipulated orthogonally the EV of options and the range associated with the possible outcomes of the gamble (gamble range, GR). For example, trials characterized by -£8 EV comprise of an option associated with a sure £8 loss and a gamble that may be associated with either -£4 and -£12 (implying a £8 GR) or with -£6 and -£10 (implying a £4 GR). Crucially, in different blocks, outcome feedback was either provided or not. The two conditions were equally consequential, as participants were instructed that an outcome was collected immediately after choice independent of whether it was shown to them or not (see Methods). Our behavioural analyses were based on comparing choice during feedback versus nofeedback trials.
Previous research has focused on how outcome experience drives learning (Barron & Erev, 2003;FitzGerald et al., 2010;Glaser et al., 2012;Hertwig et al., 2004;Jessup et al., 2008;Ludvig & Spetch, 2011;Lejarraga & Gonzalez, 2011;Wu et al., 2011;Wulff et al., 2018). Here we were not interested in learning, and we minimized any learning influence by explicitly informing participants about option amounts and probabilities. Our focus was on the role of an expectation about whether feedback will be provided or not. Hence, before each new block, participants were explicitly informed about whether feedback was given. In addition, in the task feedback was totally irrelevant for reward maximization, because each trial was independent and because the gamble probability was always 50-50 (and participants were explicitly informed about this). This allowed us to probe the role of expecting feedback per se, independent of whether feedback is also helpful for reward maximization in the future.
A critical aim of our study was to elucidate the neural processes underlying the influence of expecting feedback on choice. In the context of our task, theories of curiosity suggest that feedback reduces uncertainty and hence it is associated with epistemic value (Baldassarre & Mirolli, 2013;Friston et al., 2015;Gottlieb, 2012;Gottlieb et al., 2014;Kidd & Hayden, 2015). A possibility is that such epistemic value is reflected in activity within regions known to reflect the value of options. One of the key regions for evaluation and choice processes is the ventral striatum of the basal ganglia (for a review, see Bartra et al., 2013). An important observation is that, when a set of options is presented, activity in ventral striatum correlates with the EV of the chosen option (Bartra et al., 2013). This evidence can be integrated with theories of curiosity, inspiring the prediction that, in our task, at option presentation when a gamble is chosen the ventral striatum would respond more during feedback compared to no-feedback trials, reflecting an increased attraction for gambling during feedback trials.
Contrary to many previous studies, our design independently manipulates EV and GR. Standard models of choice would predict that the influence of EV and GR can be explained by a unique underlying factor, such as a non-linear value function as in expected utility theory (Von Neumann & Morgenstern, 1945) and prospect theory (Kahneman & Tversky, 1979), or a risk sensitivity as in riskreturn accounts (Markowitz, 1952;Sarin & Weber, 1993;Weber et al., 2004). For example, expected utility theory proposes that the mapping from reward amount to subjective value follows a concave function, an aspect proposed to be sufficient for explaining the combined influence of EV and GR on choice (Von Neumann & Morgenstern, 1945). By independently manipulating EV and GR, we tested whether, in line with these models, a single factor was sufficient to characterize the combined influence of EV and GR, or whether an explanation based on multiple factors was necessary. This allowed us to probe the specific neural substrates associated with the influence of EV and GR on choice. We examined whether the EV component was processed in the ventral striatum and ventromedial prefrontal cortex (vmPFC), two main regions implicated in processing the value of options (Bartra et al., 2013;Boorman et al., 2009;Fitzgerald et al., 2009;Hunt et al., 2012;Rigoli et al., 2016a;Strait el al., 2014). The GR component was predicted to be associated with activity in anterior insula. According to prior research, anterior insula is a key region for processing aspects of risk (Mohr et al., 2010;Payzan-LeNestour et al., 2013;Paulus et al., 2003;Preuschoff et al., 2008;Rigoli et al., 2016a;Rudorf et al., 2012). Given that, according to influential economic models (Markowitz, 1952;Sarin & Weber, 1993;Weber et al., 2004), measures of gambling variability such as GR capture the notion of risk, we predicted that GR was associated with activity in anterior insula.

Participants
Twenty-three healthy right-handed adults (13 females; aged 20-60, mean: 37, SD: 5) provided written consent to participate in the experiment. All participants had normal or corrected-to-normal vision. None had history of head injury, a diagnosis of any neurological or psychiatric condition (including any form of pathological gambling disorder, impulse control disorder or behavioural addiction), or were currently on medication affecting the central nervous system. The study was approved by the King's College London Research Ethics Committee.

Experimental paradigm and procedure
During MRI scan, participants performed a computer-based task lasting approximately 40 minutes.
On each trial (384 trials were run in total), a single monetary amount was presented on one side of the screen together with two different monetary amounts (one above the other) on the other side (sides were counterbalanced across trials). The single amount indicated a sure option and the pair of amounts indicated the two possible outcomes of a gamble. The two options always had equivalent EV. The EV was either a gain or loss (the text on the screen was coloured green and red during gain and loss trials, respectively) within the £5-£10 range (in £1 steps), with different EVs having equal frequency. The two possible outcomes of the gamble had always 0.5 probability. The difference between the two possible outcomes of the gamble (gamble range; GR) varied within the £1-£4 range (in £1 steps), with different GRs having equal frequency. EV and GR were manipulated orthogonally.
After option presentation, participants chose either option by pressing the corresponding left or right button on a keypad. In some trials (feedback condition), the outcome was presented for one second immediately after choice, followed by an intertrial interval of one second. In other trials (nofeedback condition) the outcome was not revealed, but a question mark was presented for one second immediately after choice. EV and GR varied pseudorandomly trial by trial, while feedback and no-feedback trials were grouped in alternating blocks (each including 24 trials; whether the starting block was associated with feedback or no-feedback was counterbalanced across subjects).
Note that EV, GR and feedback/no-feedback were orthogonal by design. We decided to organize the feedback and no-feedback trials in blocks as we reasoned this would help participants to remember whether a trial was followed by feedback or not. Before a transition from feedback to no-feedback condition (or vice versa) occurred, a panel on screen explicitly informed participants on whether outcome was provided or not in the upcoming block.
Before playing the task, participants received a £30 monetary endowment. At the end of the experiment, one single trial was randomly selected and the associated outcome was either added to (for gains) or subtracted from (for losses) the endowment. Crucially, the trial could be selected from both feedback or no-feedback trials, implying these conditions were equally consequential.
Data were collected at the Institute of Psychiatry, Psychology and Neuroscience at King's College London. Participants were first explicitly instructed about the task, the gamble probability, the difference between feedback and no-feedback conditions, and on how payment was derived. Inside the MRI scanner, a high-resolution structural brain scan was first recorded, followed by functional acquisition during task performance (organized in four different runs, each lasting about 10 minutes). Subjects were paid after the scanning was completed.

Behavioural analysis and computational modelling
The main goal of our behavioural analyses was to establish whether gambling varies when comparing conditions where feedback is expected versus condition where feedback is not expected.
This question can also arbitrate between different theories of choice. According to theories of anticipatory affect (Bell, 1985;Camille et al., 2004;Coricelli et al., 2005;2007;Loomes & Sugden, 1986), participants expect that one of two contrasting feelings will be elicited by feedback, namely disappointment (elicited when an outcome is worse than expected) or elation (elicited when an outcome is better than expected). This view implies that such anticipatory feelings are evoked only during feedback trials, and not during no-feedback trials. It has been postulated that anticipatory disappointment and anticipatory elation have equal weight (Loomes & Sugden, 1986). In other words, according to this view, during choice an agent anticipates a feeling of disappointment as much as a feeling of elation, with the result that these two anticipatory feelings cancel each other out. In our task, this predicts that choice will not vary during feedback versus no-feedback trials.
However, another proposal is that anticipation of disappointment usually prevails over anticipation of elation (Bell, 1985). This predicts that, in our task, individuals will gamble less during feedback compared to no-feedback trials, because expecting feedback would elicit anticipatory disappointment. An alternative perspective is endorsed by theories of curiosity (Baldassarre & Mirolli, 2013;Friston et al., 2015;Gottlieb, 2012;Gottlieb et al., 2014;Kidd & Hayden, 2015). These predict that gambling will increase during feedback trials, because feedback resolves uncertainty about which outcome has occurred in that trial.
We were also interested in exploring an aspect of decision-making which is independent of the feedback manipulation, exploiting the fact that in or task EV and GR varied independently. Previous research which has manipulated these variables Tobler et al., 2009) has relied on standard choice models. These predict that the influence of EV and GR can be explained by a unique underlying factor, such as a non-linear value function as in expected utility theory (Von Neumann & Morgenstern, 1945) and prospect theory (Kahneman & Tversky, 1979), or risk sensitivity as in risk-return accounts (Markowitz, 1952;Sarin & Weber, 1993;Weber et al., 2004). By adopting a novel analysis approach (see below), we tested whether, in line with these models, a single factor was enough to characterize the combined influence of EV and GR, or whether an explanation based on multiple factors was necessary.
To address these questions, we adopted a model-based approach by fitting several alternative models of choice behaviour and compering them. We first estimated the following logistic regression model of choice (gambling was coded as one and no gambling was coded as zero), which we will refer to as Model one: Where µ is an intercept parameter, captures the effect of the trial EV, captures the effect of the GR (this variable was rescaled to the mean), and captures the effect of the feedback/nofeedback condition (feed; coded assigning one to feedback and minus one to no-feedback). Note that EV, GR and feedback/no-feedback were orthogonal by design. To address some of our research questions and to perform some control analyses, Model one was compared against simpler models (i.e., where one or more parameters were fixed) as well as against more complex models (i.e., where additional parameters were included) (see Results, tab. 1 and tab. S1).
Model one describes choice as arising from multiple factors, because EV and GR exert independent influences on choice, and because choice is also characterized by a baseline gambling tendency (captured by the intercept μ) which is independent of EV and GR. This contrasts with accounts based on either a non-linear value function or on risk-return models, which are based on the notion that a single factor drives choice. This raises the question of whether, in our task, choice was better characterised by Model one or by non-linear value function or risk-return models. To define these non-linear value function or risk-return models, let us consider an option x associated with a set of possible outcomes, each defined by a monetary amount that can be a gain ( ) or a loss ( ), and by its probability (either or for gains and losses, respectively). We considered models of choice based on a non-linear value function (Kahneman & Tversky, 1979;Rutledge et al., 2015;Sokol-Hessner et al., 2009), in which each option value is obtained as: Where α is a free parameter capturing the curvature of the value function and λ is a free parameter capturing the weight of losses compared to gains (both parameters are larger than zero). A value of α < 1 describes a concave value function, while α > 1 describes a convex function. For λ < 1 and λ > 1, losses are weighted less and more than gains, respectively. We also considered models of choice based on risk-return (Markowitz, 1952;Sarin & Weber, 1993), in which corresponds to: Where indicates the variance across outcomes of an option x, and η is a free parameter capturing whether (and how much) variance is attractive (and can be positive or negative, indicating variance is attractive or repulsive, respectively). We assessed also other risk-return accounts that follow equation 3 except that they use the coefficient of variation  or GR instead of variance.
Models based on value function or risk-return prescribe that, after the value of the gamble and the value of the sure option are computed, the probability of gambling is obtained using logistic regression as: Note that, in our task, risk-return accounts are mathematically equivalent to logistic regression models having the variability of the gamble as single predictor. This is because, according to equation 3, the value of the sure option is = , the value of the gamble is = + , and hence the value difference is − = . From this, it is evident that the parameter is equivalent to the parameter in logistic regression.
We fitted multiple models based on value function or risk-return accounts (for some, the parameter α or η was estimated separately for gain and loss trials) (tab. 2). These were compared with Model one. In some versions, we also included an extra parameter to account for previous literature suggesting that gambling may exert an attraction or repulsion which is independent of the money at stake (Rigoli et al., 2016a;2016b). If, for example, we consider a non-linear value function model, this extra parameter ω operates during estimation of as follows: An important question is whether, like models relying on value function and risk-return, Model one can be formulated in terms of option evaluation, and here we offer such formulation. Let us assume that agents estimate the value of the sure option as = , and the value of the gamble as: The parameter captures the effect of feedback versus no-feedback condition. The parameter µ captures a gambling bias which is independent of EV and GR (higher value of µ boost the value of gambling). The parameter is analogous to the parameter η in risk-return accounts, and reports an attraction or repulsion for GR. The parameter reflects how much one relies on EV when calculating compared to , and we can assume it is bounded between zero and two. If < 1, EV is weighted less when estimating compared to ; if > 1, EV is weighted more when estimating compared to (if = 1, EV is weighted equally). We can write the value difference between the two options as follows: This is equivalent to equation one if we set = − 1. Note that can be positive or negative, and can be interpreted as reflecting whether one weights more EV when computing ( > 1) or when computing ( < 1).
Models were fitted separately for each participant. Parameters were estimated with Matlab, and specifically using the glmfit function for logistic regression models, and the fminsearchbound function for models based on value function or on risk return accounts. Parameter estimation was unconstrained except for the non-linear value function parameter α and the loss weight parameter λ, which were bounded to be positive. After parameters were estimated, the log-likelihood of the data given the model was calculated and summed across participants. This was used to derive the Bayesian Information Criterion (BIC), which approximates the posterior probability of the model given the data (the better the model the lower the BIC score).

fMRI scanning and analysis
The task was programmed using the Cogent toolbox (Wellcome Trust Centre for Neuroimaging) in Matlab. Visual stimuli were back projected onto a translucent screen positioned behind the bore of the magnet and viewed via an angled mirror. Blood oxygenation level dependent (BOLD) contrast functional images were acquired with echo-planar T2*-weighted (EPI) imaging using a Siemens Trio 3-Tesla MR system with a 16 channel head coil. The whole brain was covered by images comprising 40 interleaved 3-mm-thick sagittal slices (in-plane resolution = 3 x 3 mm; time to echo = 30 ms; repetition time = 2 s; flip angle = 75°; field of view = 240mm). The first six volumes were discarded to allow for T1 equilibration effects. T1-weighted structural images were acquired at a 1 x 1 x 1 mm resolution (time to echo = 3.016 ms; repetition time = 7.312 ms; flip angle = 11°; field of view = 270mm). Functional MRI data were analysed using Statistical Parametric Mapping (SPM) version 12 (Wellcome Trust Centre for Neuroimaging). Data preprocessing included spatial realignment, slice timing correction, normalization and smoothing. Specifically, functional volumes were realigned to the mean volume, were spatially normalized to the standard Montreal Neurological Institute (MNI) template with a 3 x 3 x 3 voxel size, and were smoothed with 8 mm Gaussian kernel. High-pass filtering with a cut-off of 128 s and AR(1)-model were applied.
Blood oxygenation level dependent (BOLD) response was modelled with a canonical hemodynamic response function and a general linear model (GLM) including, when options were presented, one stick function regressor modulated by (i) choice (a binary variable assigning one to gambling and minus one to sure choices), (ii) EV, (iii) GR, (iv) feedback (a binary variable assigning one to feedback and minus one to no-feedback condition), (v) the EV-choice interaction, (vi) the GR-choice interaction, (vii) the choice-feedback interaction, (viii) the GR-feedback interaction, and (ix) RTs (as effect of no interest). For feedback trials, two stick function regressors were also included at the time of outcome, the first for choices of the gamble (modulated by a RPE; i.e., the difference between outcome and EV), the second for choices of the sure option. The GLM also included six motion parameters as nuisance regressors. Please see SI for arguments ruling out any potential collinearity among regressors, for example among regressors at the time of option presentation and regressors at the time of outcome. SI also describes additional fMRI analyses based on a different GLM to explore on the feedback-EV and feedback-GR interaction.
Contrasts of interest were computed subject by subject, and used for second-level (between subjects) one-sample t-tests and Pearson correlations using standard summary statistic approach (in analyses reported below, age and gender were not adopted as covariates; however, similar results were obtained when age and gender were also included as covariates -results not shown). All analyses below examine the time of option presentation (the time of feedback was not the focus of the study). Statistical (small volume corrected -SVC) tests focused on the ventral striatum, vmPFC and anterior insula as ROIs. The ventral striatum and vmPFC are two primary structures involved in processing the value of options (Bartra et al., 2013;Boorman et al., 2009;Fitzgerald et al., 2009;Hunt et al., 2012;Rigoli et al., 2016a;Strait el al., 2014), while anterior insula is a key region for processing aspects of risk (Mohr et al., 2010;Payzan-LeNestour et al., 2013;Paulus et al., 2003;Preuschoff et al., 2008;Rigoli et al., 2016a;Rudorf et al., 2012). ROIs were defined a spheres having 10mm radius, and were centred on coordinates derived from previous literature (note this produced ROIs including only grey matter voxels). We were interested in examining striatum and vmPFC, given their central role in processing EV (Bartra et al., 2013). Hence, for striatum (coordinates: 12, 10, -6 and -12, 12, -6) and vmPFC (coordinates: 2, 46, -8), prior centre coordinates were extracted from a recent metanalysis on fMRI and EV encoding (see tab. 1 in Bartra et al., 2013). We were also interested in examining the role of anterior insula in processing risk. At present a metanalysis about risk processing in this region is not available in the literature. Hence, prior centre coordinates (coordinates: 30, 26, -2 and -30, 29, 1) were extracted from a recent study that used a similar design to examine aspects of decision-making under risk (Rigoli et al., 2016a). For striatum and insula, a separate ROI was considered for each hemisphere. Statistics of ROIs were SVC using a voxel-wise approach (i.e., considering each single voxel individually). In order to obtain a total family wise error (FWE) rate of p < 0.05 among all five ROIs combined, a Bonferroni correction was adopted by using p < 0.01 SVC as significance threshold applied to each ROI individually. For exploratory purposes, we also report data for other brain regions with statistics having p < 0.001 uncorrected significance (tab. S3).
An increased gambling for gains compared to losses, a lack of risk aversion for gains, and risk aversion for losses are observations which do not fit with prevailing choice models (Kahneman & Tversky, 1979;Von Neumann & Morgenstern, 1945). However, these results are consistent with studies adopting similar task designs (for an enhanced gambling for gains compared to losses, see Ludvig & Spetch, 2011;Rutledge et al., 2015; for a lack of risk aversion for gains, see Rigoli et al., 2016a;2016b; for risk aversion for losses, see Ludvig & Spetch, 2011;Rutledge et al., 2015). A possibility is that this pattern emerges when small monetary amounts are at stake as potential gains or losses (Prelec & Loewenstein, 1991).
To probe further our data, we fitted alternative models of choice and compared them (see Methods). Model one (see Methods) showed a lower BIC score compared to simpler models in which one or more parameters were set to zero (tab. 1). Furthermore, Model one was compared with more complex versions which included also a gain/loss factor (coded assigning one to gains and zero to losses), which in some models interacted with EV, GR, or feedback/no feedback (tab. 1), and models where feedback/no feedback interacted with EV or GR. These more complex models had larger BIC (tab. 1). This model comparison analysis supports Model one over alternative accounts.
Note that Model one implements as free parameter, which indicates that participants' choice varied based on feedback condition. Consistent with the ANOVA above, the parameter was significantly larger than zero (mean = 0.31; t(22) = 4.34, p < 0.001), reflecting an increased gambling for feedback versus no-feedback condition.
Model one characterizes the effect of feedback condition using a binary variable, which describes the belief of whether feedback is expected or not. However, because feedback and no-feedback conditions were organized in blocks, the feedback effect could potentially result from an influence of outcomes collected before (and not from expectations about feedback). We considered this analysing the influence of the outcome collected at the previous trial, focusing on its associated value or on its associated reward prediction error (RPE; i.e., the outcome value minus the EV). We estimated logistic regression models of choice similar to Model one in which either the value or RPE of the previous outcome was included as additional predictor (note that this predictor has a value of zero for no-feedback trials) or replaced the feedback/no-feedback binary predictor (tab. S1). In some models, we assumed different influences for gains and losses or for positive and negative RPEs. For example, in one model the previous outcome value was set to zero when it was a loss. Other models used the unsigned value or unsigned RPE associated to the previous outcome. Finally, we assessed models implementing the interaction between the value of the previous outcome (considering, for some models, its absolute value and, for other models, considering exclusively either losses or gains) and choice, the latter indicating whether a sure option or a gamble was chosen at the previous trial (note that the interaction between RPE and choice cannot be modelled, because RPE is always zero when the sure option is selected). All models implementing an influence of the previous outcome reported a BIC score larger than Model one (tab. S1), indicating it is unlikely that participants were influenced by the previous outcome. To ascertain further that the effect of feedback condition was not confounded by any influence of the previous outcome, we tested (the regressor associated with feedback condition) also for models implementing the role of outcome, and results confirmed that this parameter remained significant (e.g., for the model implementing RPE at previous trial: t(22) = 4.22, p < 0.001; results not shown for other models). Altogether, analyses on the influence of previous outcome support the notion that the effect of feedback condition depends on an expectation about whether feedback will be available or not, and it is not confounded by learning.
Next, we compared Model one against value function (Kahneman & Tversky, 1979;Von Neumann & Morgenstern, 1945) or risk-return accounts (Markowitz, 1952;Sarin & Weber, 1993;Weber et al., 2004) (see Methods). Different versions of these accounts were considered during model comparison (see Methods and tab. 2). These showed a BIC score higher than Model one, suggesting they provide a worse explanation of our data (tab. 2; see SI and tab. S2 for an analysis of simulated data which clarifies why non-linear value function or risk-return models poorly explain choice in our task). This suggests that, contrary to value function or risk-return accounts, a single factor is not enough to explain choice in our task, but that multiple factors are at play (as implicated by Model one).
Overall, Model one emerged as the best account of choice behaviour. This shows that gambling resulted from four independent components, namely a gambling bias (captured by the intercept µ), a component dependent on EV, one dependent on GR, and one reflecting the presence or absence of feedback. Note that the observation that choice behaviour was affected by these different variables characterising the different trials indicates that participants overall felt that their choices were consequential, in other words that participants were (at least to some degree) motivated during the task. Note also that the gambling bias is formally equivalent to the total gambling An important question is whether the parameters estimated for Model one are reliable. For example, can we reliably infer that a participant is less affected by feedback compared to another participant? To assess this, for each participant we randomly split trials in two groups, and Model one was estimated from each trial group. All parameters estimated from the two trial groups were correlated across participants ( fig. S2; µ: r(21) = 0.959 , p < 0.001; : r(21) = 0.955 , p < 0.001; : r(21) = 0.956, p < 0.001; : r(21) = 0.776 , p < 0.001), indicating that the parameters were reliable, and that the individual differences across participants were meaningful and not due to chance.
To assess whether the effects exerted by the different factors were similar across participants (e.g., whether participants on average gambled more for larger EVs), we performed one-sample t-tests on the estimated parameters (see above for this test regarding ). We did not find any difference gambling when feedback is provided, and show that both the EV and the GR each exert an independent influence in our gambling task.

Neuroimaging
Our computational model-based analysis allowed us to estimate, for each subject, different components driving choice behaviour in our task, namely a baseline propensity component, an EVrelated component, a GR-related component, and a feedback-related component. We characterized the neural processes associated with these. To this aim, we probed the relationship between individual differences in gambling behaviour and BOLD response. Identifying a relationship in a certain brain region for a specific component of gambling would suggest the region is involved in that component.
We fitted a GLM to BOLD data (see Methods) having one stick function regressor at the time of option presentation modulated by (i) choice (a binary variable assigning one to gambling and minus one to sure choices), (ii) EV, (iii) GR, (iv) feedback (a binary variable assigning one to feedback and minus one to no-feedback), (v) the EV-choice interaction, (vi) the GR-choice interaction, (vii) the choice-feedback interaction, and (viii) RTs (as effect of no interest). For feedback trials, two stick function regressors were also included at the time of outcome, the first for choices of the gamble (modulated by a RPE), the second for choices of the sure option (see SI for considerations ruling out any potential collinearity among regressors). All our analyses below examine the time of option presentation (the time of feedback was not the focus of the study).
We first probed the neural processes implicated in an increased gambling during feedback trials.
Previous research has shown that, when options are presented, response in ventral striatum encodes the value of the chosen option (Bartra et al., 2013). This raises the possibility of a stronger striatal response during feedback compared to no-feedback trials when a gamble is chosen, given that (according to our behavioural results) gambling is more attractive when feedback is provided.
Specifically, during feedback compared to no-feedback trials, we predicted a stronger striatal . Although this emerged only as a trend, this results suggests that the more a participant gambled for feedback compared to no feedback trials, the more striatal response for gambling versus sure choices increased during feedback compared to no-feedback trials. Note this effect occurs at option presentation and is not confounded by the outcome, since the GLM adopted includes specific regressors at outcome presentation (see SI for further analyses on this point).
To investigate the neural processes underlying an EV-dependent gambling preference, we focused on ventral striatum and vmPFC. While, when options are presented, response in ventral striatum encodes the EV of the chosen option (Bartra et al., 2013), previous research suggests that vmPFC signals a difference in EV across options (Bartra et al., 2013;Boorman et al., 2009;Fitzgerald et al., 2009;Hunt et al., 2012;Rigoli et al., 2016a;Strait el al., 2014).  (Bartra et al., 2013;Boorman et al., 2009;Fitzgerald et al., 2009;Hunt et al., 2012;Rigoli et al., 2016a;Strait el al., 2014). In our task, this predicts that, for participants who gamble more for negative EVs, the relationship between EV and vmPFC activity should be positive during sure choices, and negative during gambling. Vice versa, for participants who gamble more for positive EVs, the relationship between EV and vmPFC activity should be negative during sure choices, and positive during gambling. We investigated this analysing the EV-choice interaction term of the BOLD GLM (i.e., the term of the GLM numbered as v). Contrary to predictions based on previous literature (Bartra et al., 2013;Boorman et al., 2009;Fitzgerald et al., 2009;Hunt et al., 2012;Rigoli et al., 2016a;Strait el al., 2014), in vmPFC this interaction term was not significantly different from zero and was not correlated with an EV-related gambling propensity (p > 0.01 SVC). No effect emerged in ventral striatum too (p > 0.01 SVC).
Evidence indicates that, in humans, anterior insula appears as one of the main brain regions implicated in processing aspects of risk (Mohr et al., 2010;Payzan-LeNestour et al., 2013;Paulus et al., 2003;Preuschoff et al., 2008;Rigoli et al., 2016a;Rudorf et al., 2012). Specifically, an association has been reported between activity in this region and measures of risk linked with the probability of future outcomes (Mohr et al., 2010;Payzan-LeNestour et al., 2013;Paulus et al., 2003;Preuschoff et al., 2008;Rigoli et al., 2016a;Rudorf et al., 2012). However, so far fMRI studies on risky decisionmaking have operationalized risk in terms of outcome probability only (note that, as in our study, also in previous studies on risk outcomes were irrelevant for learning). The role of other components of risk, such as GR or similar measures of outcome variability, remains poorly understood. Our design allowed us to investigate this question. We considered the hypothesis that insula reflects the total level of expected risk, which, according to influential economic models (Markowitz, 1952;Sarin & Weber, 1993;Weber et al., 2004), increases (i) during gambling choices, (ii) when options have higher GR, and (iii) when a gamble characterized by higher GR is chosen. To test this, we first compared insula response for gambling versus sure choices, and, consistent with previous reports Previous research has reported a link between gambling preferences and insula activity (Rudorf et al., 2012). However, in previous studies insula was not recorded during task performance. Therefore, an open question is whether the risk signal found in insula during task performance is implicated in gambling preferences. To address this question, we first asked whether insula response for gambling . This suggests that, when comparing gambling and sure choices, the association between GR and insula response was stronger for participants who, at the behavioural level, preferred gambling for smaller GRs. Altogether, these results highlight a link between risk signalling in insula and risk preferences. They indicate that insula response to a specific aspect of risk (e.g., when a gamble is selected) is amplified if that aspect is perceived as aversive (e.g., if gambling is usually avoided).
In summary, our fMRI analyses identify specific neural substrates associated with the different components of gambling that guide choice behaviour in our task. When a gamble is chosen (compared to sure choices), striatal activation at option presentation was higher during feedback compared to no-feedback trials. A preference to gamble dependent on EV was related with an effect of EV on vmPFC activity. Insula response increased for gamble compared to sure choices and was associated with GR, an association that was stronger during gambling compared to sure choices. In addition, effects on insula were linked with individual gambling preferences, as the increase in insula activity for gamble compared to sure choices was stronger in participants who gambled less, and the association between GR and insula response was stronger during gambling for participants who preferred gambling with small GR.

DISCUSSION
We examined the role of expecting feedback, and we found this increased gambling. At option presentation, expecting feedback was associated with increased striatal activity when a gamble was being selected. EV and GR exerted independent influences on gambling in a way that was inconsistent with both non-linear value function (Kahneman & Tversky, 1979;Von Neumann & Morgenstern, 1945) and risk-return accounts (Markowitz, 1952;Sarin & Weber, 1993;Weber et al., 2004). The effects of EV and GR on choice mapped to distinct neural processes reflected in vmPFC and in anterior insula activity at option presentation, respectively.
Our observation of an increased gambling during feedback trials is hard to explain as being the consequence of anticipatory disappointment (Bell, 1985). However, this finding fits with current models of curiosity (Baldassarre & Mirolli, 2013;Friston et al., 2015;Gottlieb, 2012;Gottlieb et al., 2014;Kidd & Hayden, 2015). Epistemic value is a central concept of curiosity theories (e.g., Friston et al., 2015) and corresponds to a form of value which favours information seeking (and hence uncertainty reduction) even when this is not helpful for reward collection. A critical aspect of our task is that feedback was not instrumental, in other words it did not facilitate reward collection in the future. Hence, our findings suggest that epistemic value associated with expecting feedback facilitates gambling.
Previous research has examined decision-making under risk in contexts where agents are not explicitly informed about possible outcomes and probabilities, but have to learn these from their past experience (Barron & Erev, 2003;FitzGerald et al., 2010;Glaser et al., 2012;Hertwig et al., 2004;Jessup et al., 2008;Ludvig & Spetch, 2011;Lejarraga & Gonzalez, 2011;Wu et al., 2011;Wulff et al., 2018). Crucially, in this research feedback was instrumental for learning and for reward maximization, as it allowed participants to acquire knowledge about outcomes and probabilities. The fact that learning was engaged implies that previous studies were unable to assess whether individuals are attracted by feedback information even when this information is irrelevant for performance. On the contrary, given that participants were explicitly informed about outcomes and probabilities, our task does not involve any learning (this is supported by analyses demonstrating that the previous outcomes did not influence choice). This allowed us to show that individuals are attracted by feedback information even when this information is irrelevant. Our findings also help in interpreting previous observations (Ludvig & Spetch, 2011). Comparing decisions from experience (i.e., based on learning) versus decisions from instruction (the latter occurring when option consequences are described explicitly), an increased gambling has been reported in a task analogous to the one used here (Ludvig & Spetch, 2011). Our study raises the possibility that this previous observation could be explained by an attractiveness of feedback, irrespective of any learning.
Recording fMRI data during task performance allowed us to shed light on the neural mechanisms underlying the effect of expecting feedback on choice. Increased gambling during feedback trials reflected a higher value placed on gambling in this condition. This increased value was expressed in the ventral striatal response at option presentation -at a time when this region encodes the value of the chosen option (Bartra et al., 2013). Previous literature has asked whether the ventral striatum is implicated in processing the value of information. An indirect evidence of this is the observation that, when an individual is waiting for an answer to a question about general knowledge, activity in ventral striatum increases with the reported level of curiosity (Kang et al., 2009). However, given that this previous study did not require any choice behaviour, it remains unclear whether reported curiosity reflects subjective value. By adopting a choice task, our study provides direct evidence that the ventral striatum processes a value of information which motivates choice behaviour. Another study found the striatum to encode a novelty value signal, which favours exploration of newly presented options in a decision-making task (Wittmann et al., 2008). However, in that study exploration was instrumental to reward maximization, as it allowed participants to familiarize with novel options and hence to maximize reward collection in the long run. On the contrary, our study relies on a task where feedback information was unhelpful for obtaining more reward in the future.
Hence our findings highlight an involvement of the ventral striatum in processing a value of information in conditions where information is irrelevant for reward maximization. This signal in the striatum fits with the notion of epistemic value (e.g., Friston et al., 2015), corresponding to a form of value which favours information seeking even when this is not helpful for reward collection.
Classical theories of decision-making under risk based on value function (Kahneman & Tversky, 1979;Von Neumann & Morgenstern, 1945) or risk-return accounts (Markowitz, 1952;Sarin & Weber, 1993;Weber et al., 2004) offered a relatively poor explanation of the choice data in our task. First, as already highlighted by previous research using similar paradigms (Rigoli et al, 2016a;2016b), these models were unable to capture a baseline gambling preference (corresponding to the intercept of the logistic regression model) independent of EV or GR. Second, they were unable to account for the fact that EV and GR each exerted an independent influence on gambling. To our knowledge, this is the first demonstration that, at least in selected tasks, the models based on value function or risk return fail to describe the impact of EV and GR (or similar indexes of risk). The data were better explained by a simple logistic regression account comprising a gambling bias, an EVrelated influence, and a GR-related influence as factors (in addition to the effect of feedback condition). This model is analogous to previous accounts of multi-attribute decisions (EV and GR corresponding to two distinct attributes) in which each attribute contributes independently to choice (Tsetsos et al., 2016).
Activity in vmPFC correlated positively with EV for participants who gambled more for negative EVs, and correlated negatively with EV for participants who gambled more for positive EVs. Previous research suggests that vmPFC response signals the value difference between the chosen and unchosen option (Bartra et al., 2013;Boorman et al., 2009;Fitzgerald et al., 2009;Hunt et al., 2012;Rigoli et al., 2016a;Strait el al., 2014). Contrary to this, in our task the relationship between the effect of EV on vmPFC response and an EV-related gambling preference did not differ during gambling and sure choices. Instead, our results suggest vmPFC signalled a difference between the sure option and the gamble, independent of which option was eventually chosen. Note that, contrary to most previous studies, in our task sure options and gambles represent two clear distinct option categories. A possibility is that, when clear option categories can be identified, vmPFC activity anchors to one category, in our case the sure option. As a result, independent of the option eventually selected, activity in vmPFC would signal the difference between the reference option category and other categories, in our case corresponding to the difference between the sure option and the gamble. This offers a testable hypothesis of vmPFC activity that could be explored in the future.
Moreover, response in this region correlated with GR, and this effect was stronger during gambling compared to sure choices. In addition to encoding aspects of risk related with outcome probability (Mohr et al., 2010;Payzan-LeNestour et al., 2013;Paulus et al., 2003;Preuschoff et al., 2008;Rigoli et al., 2016a;Rudorf et al., 2012), this demonstrates that insula also processes information about outcome variability. In summary, these findings show that insula signals an overall risk expectancy integrating information about whether an option is probabilistic or not and about the variability of its possible outcomes. Although previous research has shown a link between individual risk preferences and insula response (Rudorf et al., 2012), this region was not recorded directly during decision-making. Therefore, previous research has left open the question whether insula activity during a gambling task is linked with individual strategies adopted in that task. We investigated this question and found that participants with a low gambling propensity showed a stronger insula response when comparing gambling and sure choices. Moreover, when comparing gambling and sure choices, participants who preferred gambling with low GR exhibited a stronger relationship between insula response and GR. Altogether, these results highlight a connection between insula response during task performance and gambling preferences. They indicate that the insula response to a specific aspect of risk (e.g., when a gamble is selected) is amplified if that aspect is perceived as aversive (e.g., if gambling is usually avoided). For example, participants who gamble less (i.e., those exhibiting a decreased average gambling propensity) may avoid gambling because they perceive it as an aversive option. According to our data, these participants also tend to show stronger insula response during gambling, which is putatively aversive. Hence, insula activity may reflect aspects of risk especially when these aspects are perceived as aversive.
In summary, our study shows that EV and GR each exerted an independent impact on gambling. This observation is difficult to reconcile with traditional value function and risk-return models of choice.
Signalling of EV and GR was segregated in the brain, as these quantities were associated with activity in vmPFC and insula, respectively. These results supports the notion that, at least in certain decision contexts, EV and GR are evaluated independently to form choice. A central finding of our study is that gambling increases when feedback is provided, an effect associated with increased striatal response during gambling when comparing feedback and no-feedback trials.

DECLARATION OF INTEREST
Declarations of interest: none.  Task paradigm and behavioural results. A: Description of the task, illustrating an example of two consecutive trials. After an intertrial interval (ITI) of one second, a sure option (e.g., £6) and a 50-50 gamble (e.g., having either £10 or £2 as possible outcomes) are presented. In some blocks (as in the example illustrated here), outcome feedback is provided (e.g., £6) for one second after choice, while in other blocks outcome feedback is not provided and a new trial starts immediately after choice. The figure illustrates also the following trial, in this instance offering a choice between a sure £7 loss and a gamble associated with either a £9 or £5 loss. Note the text is coloured green and red for trials involving gains and losses, respectively. Across trials, we manipulated the expected value (EV; note the two options always had equivalent EV), corresponding to £6 and -£7, respectively, for the two trials in the example. We also manipulated orthogonally the gambling range (GR), corresponding to £8 and £4, respectively, for the two trials in the example. B: Gambling proportion for the different experimental conditions. Statistical tests indicate a higher gambling for feedback versus no-feedback (F(1,22) = 17.61, p < 0.001) and for gain versus loss (F(1,22) = 5.54, p = 0.028), with no interaction (F(1,22) = 0.21, p = 0.653). C: Relationship between the EV-related gambling parameter and the GR-related gambling parameter across participants, which is not statistically significant (r(21) = -0.058, p = 793). D: Gambling proportion for trials characterized by small GR (in which GR = 2), medium GR (in which GR = 4 or GR = 6), and large GR (in which GR = 8), plotted separately for participants having a positive GR-related gambling parameter (n = 13; in yellow) and participants having a negative GR-related gambling parameter (n = 10; in cyan). E: Gambling proportion for trials characterized by large losses (in which EV = -10, EV = -9, or EV = -8), small losses (in which EV = -7, EV = -6, or EV = -5), small gains (in which EV = 5, EV = 6, or EV = 7), and large gains (in which EV = 8, EV = 9, or EV = 10), plotted separately for participants having a positive EV-related gambling parameter (n = 18; in blue) and participants having a negative EV-related gambling parameter (n = 5; in red).  Relationship between vmPFC activity at option presentation and EV-related preference. A: Voxels within vmPFC in which, across participants, the effect of EV on BOLD response correlated negatively with the EVrelated gambling parameter . For display purposes, voxels characterized by a statistic associated with p < 0.001 uncorrected are highlighted in red. B: The relationship between the effect of EV on BOLD response and the EV-related gambling parameter is plotted for the peak activation voxel within vmPFC, where a trend towards a negative correlation emerges (12, 41, -11; Z = 3.13; r(21) = -0.616; p = 0.033 SVC). C: BOLD response in vmPFC (12, 41, -11) for trials characterized by large losses (in which EV = -10, EV = -9, or EV = -8), small losses (in which EV = -7, EV = -6, or EV = -5), small gains (in which EV = 5, EV = 6, or EV = 7), and large gains (in which EV = 8, EV = 9, or EV = 10), plotted separately for participants having an EV-related gambling parameter larger than 0.1, (n = 7; in blue), participants having a EV-related gambling parameter larger than zero and smaller than 0.1 (n = 9; in cyan), and participants having a EV-related gambling parameter smaller than zero (n = 5; in red). The BOLD response corresponds to the beta weight of a GLM including specific regressors for each EV bin. This GLM was estimated for display purposes. Relationship between anterior insula response at option presentation and risk. A: In red, voxels within insula in which activity increased for gambling compared to sure choices (left: -33, 17, -8; Z = 3.59; p = 0.010 SVC; right: 33, 20, -5; Z = 4.71; p < 0.001). In yellow, voxels within insula in which activity correlated with GR (left: -33, 23, -5; Z = 3.66; p = 0.008 SVC; right: 42, 20, -5; Z = 3.68; p = 0.008 SVC). In green, voxels within insula in which the effect of GR on BOLD activity was stronger during gambling compared to sure choices (left: -30, 26, -8; Z = 3.64; p = 0.008 SVC; a trend emerged on the right hemisphere: 45, 32, -2; Z = 3.03; p = 0.042 SVC). In orange, voxels in which these effects overlap. For display purposes, voxels characterized by a statistic associated with p < 0.001 uncorrected are highlighted. B: BOLD response in anterior insula (-30, 26, -8) for trials characterized by small GR (in which GR = 2), medium GR (in which GR = 4 or GR = 6), and large GR (in which GR = 8), plotted separately for gambling (in green) and sure choices (in orange). The BOLD response corresponds to the beta weight of a GLM including specific regressors for each GR bin. This GLM was estimated for display purposes. In red, voxels within insula in which activity for gambling versus sure choices correlated negatively with a baseline gambling bias µ across participants. In green, voxels within insula in which, when comparing gambling and sure choices, the association between GR and insula response correlated inversely with the GR-related gambling propensity . In dark green, voxels in which these effects overlap. For display purposes, voxels characterized by a statistic associated with p < 0.001 uncorrected are highlighted. B: The relationship between the effect of gambling versus sure choices on BOLD response and a baseline gambling bias µ is plotted for the peak activation voxel within insula (39, 17, -8; Z = 3.94; r(21) = -0.728; p = 0.003 SVC). C: The relationship between the effect of GR during gambling versus sure choices (GR-choice interaction) on BOLD response and a GR-related gambling propensity is plotted for the peak activation voxel within insula, where a trend towards significance emerges (-33, 29, -5; Z = 3.39; r(21) = -0.655; p = 0.016 SVC).