{'fraction_non_alphanumeric': 0.027932960893854747, 'fraction_numerical': 0.01097366320830008, 'mean_word_length': 5.468387096774194, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Inferring Subjective Prior Knowledge: An Integrative Bayesian Approach

Inferring Subjective Prior Knowledge: An Integrative Bayesian Approach Sean Tauber (sean.tauber@uci.edu) Mark Steyvers (mark.steyvers@uci.edu) Department of Cognitive Sciences, University of California, Irvine Irvine, CA 92697 USA Abstract The standard approach to Bayesian models of Cognition (also known as rational models) requires researchers to make strong assumptions about people’s prior beliefs. For example, it is often assumed that people’s subjective knowledge is best represented by “true” environmental data. We show that an integrative Bayesian approach—combining Bayesian cognitive models with Bayesian data analysis—allows us to relax this assumption. We demonstrate how this approach can be used to estimate people’s subjective prior beliefs based on their responses in a prediction task. Keywords: Bayesian modeling; rational analysis; cognitive models; Bayesian data analysis; Bayesian inference; knowledge representation; prior knowledge Introduction In the standard approach to Bayesian models of Cognition (also referred to as rational models), researchers make strong assumptions about people’s prior beliefs in order to make predictions about their behavior. These models are used to simulate the expected behavior—such as decisions, judgments or predictions—of someone whose computational-level solution to a cognitive task is well described by the model. Analysis of Bayesian models of cognition usually involves a qualitative comparison between human responses and simulated model predictions. For an overview of Bayesian models of cognition see Oaksford and Chater (1998); but also see Mozer, Pashler, and Homaei (2008); and Jones and Love (2011) for a critique. As an alternative to the standard approach, we present an integrative Bayesian approach that allows us to relax the assumptions about people’s prior beliefs. This approach is motivated by previous efforts to infer subjective mental representations (Lewandowsky, Griffiths, & Kalish, 2009; Sanborn & Griffiths, 2008; Sanborn, Griffiths, & Shiffrin, 2010) and more specifically to combine Bayesian models of cognition and Bayesian data analysis (Huszar, Noppeney & Lengyel, 2010; Lee & Sarnecka, 2008). The integrative approach allows us to use people’s responses on a cognitive task to infer posterior distributions over the psychological variables in a Bayesian model of cognition. It also allows us to estimate probabilistic representations of people’s subjective prior beliefs. We recently applied this approach to a Bayesian cognitive model of reconstructive memory (Hemmer, Tauber, & Steyvers, in prep). We estimated individuals’ subjective prior beliefs about the distribution of people’s heights based on their responses in a memory task. The technical requirements for integrated Bayesian inference were simplified because the posterior distribution, based on inference in the cognitive model, had a simple Gaussian form. This made it straight forward to define individuals’ responses as Gaussian distributed random variables in an integrated Bayesian model. In this study, we develop a method for applying integrated Bayesian inference that does not require the posterior of the cognitive model to have a simple parametric form. We apply this method to a Bayesian cognitive model for predictions that was developed by Griffiths and Tenenbaum (2006). Their Bayesian model of cognition was a computational-level description of how people combine prior knowledge with new information to make predictions about real-world phenomena. They asked participants to make a series of predictions about duration or extent that were similar to the following examples: If you were assessing the prospects of a 60-year-old man, how much longer would you expect him to live? If you were an executive evaluating the performance of a movie that had made $40 million at the box office so far, what would you estimate for its total gross? All of the questions used by Griffiths and Tenenbaum (2006) were based on real-world phenomena such as, life spans, box office grosses for movies, movie runtimes, poem lengths and waiting times. Their assumption was that people make predictions about these phenomena based on prior beliefs that reflect their true extents or durations in the real world. Although it is possible that people’s beliefs about these phenomena are tuned to the environment, this assumption cannot be used to explain how people make similar sorts of predictions about counterfactual phenomena that have no true statistics in the environment. For example, consider the following question: Suppose it is the year 2075 and medical science has advanced significantly. You meet a man that is 60 years old. To what age will this man live? There is no “true” answer to this question and therefore no environmental data is available. This creates a problem for a Bayesian model of cognition that requires environmental data in order to make predictions.