{'fraction_non_alphanumeric': 0.026331538001196888, 'fraction_numerical': 0.01436265709156194, 'mean_word_length': 5.554248366013072, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Assessing the Efficacy of Transitional Probabilities for Learning Syntactic Categories

Assessing the Efficacy of Transitional Probabilities for Learning Syntactic Categories Erin Conwell (Erin_Conwell@brown.edu) Brown University Department of Cognitive and Linguistic Sciences, Box 1978 Providence, RI 02912 USA Benjamin J. Balas (bjbalas@mit.edu) Massachusetts Institute of Technology Department of Brain and Cognitive Sciences, 43 Vassar Street Cambridge, MA 02139 USA Artificial language studies demonstrate that learners are highly sensitive to the statistics of the language that they hear and can use that information to find word boundaries and learn word classes (Gerken, Wilson & Lewis, 2005; Gomez & Lakusta, 2004; Saffran, Aslin & Newport, 1996). Models of language learning indicate that the language that children hear appears to contain a number of cues that could be useful for learning syntactic categories (Mintz, Newport & Bever, 2002; Monaghan, Chater & Christiansen, 2005; Redington, Chater & Finch, 1998). However, these two lines of work inform one another to a limited degree. That is to say that while our understanding of infants’ linguistic abilities has advanced considerably, this knowledge is not used to inform models of category learning. Likewise, although recent models of category learning are very successful at distinguishing between grammatical categories, little empirical research is conducted to determine whether infants can make use of the kinds of cues that these models exploit (for exceptions, see Mintz, 2003; 2006; Shi, Morgan & Allopenna, 1998 and Shi, Werker & Morgan, 1999). Improved communication between these two lines of research is critical. For a model of language acquisition to be psychologically plausible, it must make use of information that is actually available to a language learner. Conversely, infants may show evidence of certain linguistic abilities, but if they are to use them for language acquisition, these abilities must be relevant to learning not only the carefully controlled language they hear in the laboratory, but also the uncontrolled corpus of speech that they hear in the real world. Our model evaluates whether the kinds of statistics that infants track in laboratory studies are useful for the acquisition of grammatical categories over a natural speech corpus. While function words are often absent from children’s earliest language productions, studies of infants’ language perception indicate that they are sensitive to the syntactic properties of function words from a very early age (Shady, 1996; Soderstrom, White, Conwell & Morgan, in press), which suggests that these words could play a critical role in the acquisition of more advanced syntax. Hohle, Weissenborn, Kiefer, Schulz and Schmitz (2004) Abstract While research on both infant language abilities and the informativeness of natural language for the formation of grammatical categories has advanced considerably, the extent to which these two fields inform each other is limited. To address this issue, we ask whether tracking transitional probabilities, a skill which infants are known to apply to language learning, is useful for learning grammatical categories from natural child-directed speech. We systematically remove subsets of the data to assess the relative contributions of several potential sources of information. Our analysis finds that immediately following a high frequency function word provides considerable information about whether a word is a noun or a verb. However, in unsupervised clustering, this information alone does not result in highly accurate categorization of nouns and verbs. Keywords: Language learning; grammatical categories; unsupervised learning; supervised learning. Introduction Central to an understanding of how children learn language is a theory of how they learn the syntactic categories of individual lexical items. Early work on this topic concluded that children must have an innate predisposition to learn categories such as noun and verb as well as a set of innate rules linking individual words to these categories via meaning (e.g., Pinker, 1984). Such theories claim that children cannot possibly learn grammatical categories from syntactic distribution because the stimulus is impoverished and noisy (Chomsky, 1965). Not only do children not receive enough positive evidence to support category learning, they also hear words used across category boundaries or without any syntactic support at all (e.g., isolated words). This characterization of the input to children’s language learning presents a grim outlook for the possibility that children, like adult linguists, categorize words based on their syntactic privileges. However, empirical research with both adult and child language learners, as well as closer examination of the language children hear, suggests that these theories may underestimate both the abilities of language learners and the informativeness of a child’s linguistic environment.