{'fraction_non_alphanumeric': 0.01634877384196185, 'fraction_numerical': 0.0, 'mean_word_length': 5.444444444444445, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
How Many Subjects Show the Effect? A Sharp Lower Bound Estimator with Examples from Behavioral Economics

Computing the average ‘effect’, the difference in means of the dependent variable between conditions, is by far the most common way of analyzing experimental data. In many settings, however, such comparison is incomplete if not misleading with respect to the question of interest. An alternative is to examine the share of subjects exhibiting an effect. This paper puts forward an assumption-free and computationally efficient algorithm that computes the sharp lower-bound for that statistic. Its use and usefulness are demonstrated reanalyzing data from behavioral economics experiments. Lower bounds for the share of subjects behaving contrary to standard economic theory in those experiments are shown to be sufficiently high and estimated with enough precision to be considered informative. Furthermore, experiments with non-significant differences in means and even with reversals from predicted effects are shown to have an informatively high lower bound of subjects behaving as predicted.