{'fraction_non_alphanumeric': 0.025137111517367458, 'fraction_numerical': 0.002742230347349177, 'mean_word_length': 5.400584795321637, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
How many choice-situations are optimal? Investigating ordering effects – the impact of the number of choice situations on individual preference and scale heterogeneity using the G-MNL model

The optimal number of choice-situations that are presented to respondents of a choice experiment is an important research question. More choice-situations per respondent increase the amount of available information, however, treating repeated choice data in the same way as cross-sectional data may lead to biased standard errors of the estimated parameters. So far there were no methods allowing to control for preference and scale heterogeneity at the same time. This is an important issue, since some empirical studies reported significant influence of increasing the number of choice-situations on cognitive processing dynamics, and utility function’s error term (or scale) in particular. It has been postulated that these ordering effects such as learning and fatigue, can in some cases result in a U-shaped relationship between error term’s variance and the number of choice-situations per respondent. We utilized the Generalized Multinomial Logit Model (G-MNL), which we extended to accommodate observable scale heterogeneity, as a function of choice-situation number. This approach allows to investigate learning and fatigue effects, while accounting for both preference- and scale-heterogeneity across respondents and choice-situations. We demonstrate how to apply this approach on a high-quality representative sample of 1000+ respondents. The CAPIs were based in the context of environmental protection – forest management in Poland. We used counterbalanced Bayesian efficient design to control for the effects of anchoring, framing and strategic behaviour. Our results provide evidence that scale can indeed be non-constant, and choice-situation number specific, what leads to different share of deterministic part of utility function, i.e. the precision with which respondents make their choices. We observed of significant institutional learning and some value learning effects. Interestingly, we did not observe any fatigue effects in the 26 choice-situations each respondent faced.