{'fraction_non_alphanumeric': 0.027170311464546057, 'fraction_numerical': 0.0145791915175613, 'mean_word_length': 5.250517598343685, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Statistical Learning, Inductive Bias, and Bayesian Inference in Language Acquisition

Language acquisition is a problem of induction: the child learner is faced with a set of specific linguistic examples and must infer some abstract linguistic knowledge that allows the child to generalize beyond the observed data, i.e., to both understand and generate new examples. Many different generalizations are logically possible given any particular set of input data, yet different children within a linguistic community end up with the same adult grammars. This fact suggests that children are biased towards making certain kinds of generalizations rather than others. The nature and extent of children's inductive bias for language is highly controversial, with some researchers assuming that it is detailed and domain-specific (e.g., Chomsky 1973, Baker 1978, Chomsky 1981, Huang 1982, Fodor 1983, Bickerton 1984, Lasnik & Saito 1984, Gleitman & Newport 1995) and others claiming that domain-general constraints on memory and processing are sufficient to explain the consistent acquisition of language (e.g., Elman, Bates, Johnson, Karmiloff-Smith, Parisi, & Plunkett 1996, Sampson 2005). In this chapter, we discuss the contribution of an emerging theoretical framework called Bayesian learning that can be used to investigate the inductive bias needed for language acquisition. In the Bayesian view of learning, inductive bias consists of a combination of hard and soft constraints. Hard constraints make certain grammars impossible for any human to acquire; in the language of Bayesian modeling, these impossible grammars are outside the learner's hypothesis space. Grammars inside the hypothesis space are learnable given the right input data, but they may not all be equally easy to learn. Soft constraints, implemented in the form of a probability distribution over the hypothesis space, mean that the learner will be biased towards certain of these grammars more than others. A "difficult" (low-probability) grammar can be learned, but will require more evidence (input data favoring this grammar) in order to be learned. In the absence of such evidence, the child will instead acquire a high-probability grammar that is also compatible with the input. Under this view of learning, the central question of language acquisition is to determine what the hard and soft constraints are. A key assumption is that learners have access to domain-general statistical learning mechanisms that closely approximate the rules of probability theory. Given a particular set of input data, these probabilistic learning mechanisms then allow learners to converge on a grammar that is both compatible with the data and has high probability in the hypothesis space. In this sense, the grammar is the optimal choice, given the data. This notion of optimization arises from the ties between Bayesian modeling and the tradition of rational analysis in cognitive science (Chater & Oaksford, 1999), which focuses on the adaptation of the