{'fraction_non_alphanumeric': 0.03699260147970406, 'fraction_numerical': 0.017196560687862426, 'mean_word_length': 5.0192539109506615, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A Connectionist Investigtion of Linguistic Arguments from the Poverty of the Stimulus: Learning the Unlearnable

A Connectionist Investigation of Linguistic Arguments from the Poverty of the Stimulus: Learning the Unlearnable John D. Lewis (jlewis@crl.ucsd.edu) Department of Linguistics, McGill University; 1085 Dr. Penfield Avenue Montreal, Quebec H3A1A7 Canada Center for Research in Language, UC San Diego; 9500 Gilman Dr. La Jolla, CA 92093-0526 USA Jeffrey L. Elman (elman@crl.ucsd.edu) Center for Research in Language, UC San Diego; 9500 Gilman Dr. La Jolla, CA 92093-0526 USA Abstract Based on the apparent paucity of input, and the non- obvious nature of linguistic generalizations, Chomskyan linguists assume an innate body of linguistically detailed knowledge, known as Universal Grammar (UG ), and at- tribute to it principles required to account for those “prop- erties of language that can reasonably be supposed not to have been learned” (Chomsky, 1975). A definitive ac- count of learnability is lacking, but is implicit in exam- ples of the application of the logic. Our research demon- strates, however, that important statistical properties of the input have been overlooked, resulting in UG being credited for properties which are demonstrably learnable; in contradiction to Chomsky’s celebrated argument for the innateness of structure-dependence (e.g. Chomsky, 1975), a simple recurrent network (Elman, 1990), given input modelled on child-directed speech, is shown to learn the structure of relative clauses, and to generalize that structure to subject position in aux-questions. The result demonstrates that before a property of language can rea- sonably be supposed not to have been learned, it is neces- sary to give greater consideration to the indirect positive evidence in the data and that connectionism can be invaluable to linguists in that respect. Introduction Chomskyan linguists argue that language acquisition cannot strictly be a matter of learning the child’s tar- get grammar is “hopelessly underdetermined by the frag- rather mentary evidence available” (Chomsky, 1968) it must rest on a set of innate linguistic principles; the goal of the Chomskyan linguist is to determine the con- tents of this set, known as Universal Grammar ( UG ). The idea is to attribute to UG all and only the princi- ples required to account for those “properties of lan- guage that can reasonably be supposed not to have been learned” (Chomsky, 1975). Learning theory is thus of central importance to the enterprise, but, oddly, a defini- tive account of the notion of learning that Chomskyans adopt is lacking, and is given only implicitly in the examples of the principles attributed to UG . Statisti- cal approaches, however, and the notions of generaliza- tion and analogy have been explicitly rejected as irrel- evant (Chomsky, 1975). In this paper we demonstrate that UG has been that this rejection is a serious error attributed with principles to account for properties of lan- guage that are demonstrably learnable from the statistical properties of the input. Chomsky’s celebrated argument for the innateness of the principle of structure-dependence (Chomsky, 1975) serves as an example. Chomsky claims that, during the course of language acquisition, children entertain only hypotheses which respect the abstract structural organization of language, though the data may also be consistent with structure-independent hypotheses, i.e. relationships over utterances considered only as linearly ordered word sequences. As support for this claim, Chomsky notes that though questions like (1) are apparently absent in the child’s input, questions like (2) are never erroneously produced a claim subsequently 1) Is the man who is smoking crazy? 2) * Is the man who smoking is crazy? empirically tested and substantiated by Crain and Nakayama (1987, also see Crain 1991). Chomsky sug- gests that it is reasonable to suppose that children de- rive aux-questions from declaratives, and exposed to only simpler structures, might hypothesize either of two sorts of rules: a structure-independent rule i.e. move or the correct structure-dependent rule. the first ‘is’ Chomsky claims that “cases that distinguish the hy- potheses rarely arise; you can easily live your whole life without ever producing a relevant example to show that you are using one hypothesis rather than the other one” (Piatelli-Palmarini, 1980). The fact that children do not produce questions like (2), despite that the correct rule is supposedly more complex, and that the learner might not encounter the relevant evidence leads Chom- sky to suggest that “the only reasonable conclusion is that UG contains the principle that all such rules must be structure-dependent” (Chomsky, 1975). As a number of researchers have noted, however, there are several weaknesses in this argument. Slobin (1991), for instance, points out that the conclusion rests on the assumption that aux-questions are derived from declar-