{'fraction_non_alphanumeric': 0.02510460251046025, 'fraction_numerical': 0.007267121779343757, 'mean_word_length': 5.424328147100424, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Hypothesis space checking in intuitive reasoning

Hypothesis Space Checking in Intuitive Reasoning Christopher D. Carroll (cdcarroll@gmail.com) Department of Psychology, Carnegie Mellon University 5000 Forbes Ave., Pittsburgh, PA 15213 Charles Kemp (ckemp@cmu.edu) Department of Psychology, Carnegie Mellon University 5000 Forbes Ave., Pittsburgh, PA 15213 Abstract Hypothesis space checking The process of generating a new hypothesis often begins with the recognition that all of the hypotheses currently under consideration are wrong. While this sort of falsification is straightforward when the observations are incompatible with each of the hypotheses, an interesting situation arises when the observations are implausible under the hypotheses but not incompatible with them. We propose a formal account, inspired by statistical model checking, as an explanation for how people reason about these probabilistic falsifications. We contrast this account with approaches such as Bayesian inference that account for hypothesis comparison but do not explain how a reasoner might decide that the hypothesis space needs to be expanded. Figure 1 illustrates the kind of situation where hypothesis space checking may be required. There is a universe U of possible explanations for the given observations, but the hypotheses available to the reasoner fall within a hypothesis space H that is a proper subset of U. It is possible, of course, that the true explanation is not in H; the ability to determine whether this is the case would be useful. Keywords: hypothesis testing; model checking; surprise Introduction Many modern scientific disciplines are characterized by strange and unintuitive theories that previous generations of scientists never would have imagined. On a less dramatic scale, people often generate inventive explanations in their everyday lives. The existence of these unintuitive theories and inventive explanations raises an interesting question: how are these new theories and explanations discovered? In many cases, the process of generating a new hypothesis starts when the reasoner decides that all of the hypotheses currently under consideration are wrong. In some cases, the available evidence is incompatible with every hypothesis under consideration, and this decision is straightforward. In other cases, however, the available evidence is implausible under, but not strictly incompatible with, the hypotheses. In cases like these, a reasoner may engage in hypothesis space checking to decide whether the hypothesis space is adequate or needs to be expanded. Although psychologists have explored many approaches to hypothesis testing, most of these approaches are unable to account for hypothesis space checking. Bayesian accounts, for instance, are able to specify the relative strength of a hypothesis within the hypothesis space, but they do not provide criteria for evaluating the hypothesis space itself. Statisticians, however, have developed various measures that quantify the extent to which observations are surprising under a given hypothesis or hypothesis space. In this paper, we investigate the possibility that formal measures of this kind can help to explain how people decide that all of the hypotheses in their current hypothesis space are probably wrong. Figure 1: The universe U includes all possible hypotheses, and hypothesis H is the subset of these hypotheses that are currently available to the reasoner. In principle, the adequacy of H could be evaluated by computing whether the available observations are better explained by hypotheses that lie within or outside H. Bayesian inference provides one way to formalize this sort of comparative hypothesis testing. Bayes’ theorem establishes that given the observed data d, the odds that H contains the true explanation are: Equation 1 shows that the probability that � contains the true explanation depends on (1) the relative probabilities of and (2) the the data under � and under its complement � relative prior probabilities of � and � Although Equation 1 is appealing in principle, it is consists of hypotheses impossible to apply. Given that � ) will be that are unavailable to the reasoner, the term �(�|� impossible to compute (Earman, 1990, Ch. 7; Salmon, 1990). Consider the problem faced by a Newtonian physicist attempting to explain the anomalous precession of Mercury’s perihelion. Although the physicist might be able to estimate �(�|�) by considering various Newtonian ) has a paradoxical flavor: explanations, estimating �(�|�