text:
Connectionism in Linguistic Theory
Summary and Keywords
Connectionism is an important theoretical framework for the study of human cognition and behavior. Also known as Parallel Distributed Processing (PDP) or Artificial Neural Networks (ANN), connectionism advocates that learning, representation, and processing of information in mind are parallel, distributed, and interactive in nature. It argues for the emergence of human cognition as the outcome of large networks of interactive processing units operating simultaneously. Inspired by findings from neural science and artificial intelligence, connectionism is a powerful computational tool, and it has had profound impact on many areas of research, including linguistics. Since the beginning of connectionism, many connectionist models have been developed to account for a wide range of important linguistic phenomena observed in monolingual research, such as speech perception, speech production, semantic representation, and early lexical development in children. Recently, the application of connectionism to bilingual research has also gathered momentum. Connectionist models are often precise in the specification of modeling parameters and flexible in the manipulation of relevant variables in the model to address relevant theoretical questions, therefore they can provide significant advantages in testing mechanisms underlying language processes.
Keywords: connectionism, neural networks, parallel distributed processing (PDP), language development, language processing, bilingualism, simple recurrent network (SRN), self-organizing map (SOM), DevLex models
1. Organization of the Article
Computational models have played a vital role in understanding human cognitive and linguistic behaviors. They offer particular advantages in dealing with complex interactions between variables that are often intertwined in natural language learning situations, because researchers can systematically bring target variables under tight experimental control to test theoretically relevant hypotheses (McClelland, 2009). Since 1980s, a specific class of computational models has been proven to be particularly useful to language studies. This is the class of models that are inspired by connectionism, Parallel Distributed Processing (PDP), or artificial neural networks.
The goal of this article is to guide the readers through some important issues in connectionism, and to give an integrative review on how connectionist models can be used effectively in language studies. In what follows, I will first introduce the basic concepts and philosophy of connectionist models and then describe some basic types of models along with some methodological considerations for constructing them. An overview of several connectionist language models will then be given, and the focus will be on a couple of influential ones in both monolingual and bilingual areas. I will also provide a critical analysis of a couple of challenges that face researchers in the field. The article will conclude with some further readings and online resources that will be useful for new researchers.
2. Basic Concepts
Some basic ideas of connectionism can be traced back to the 1940s, at the dawn of both neuroscience and computer science. However, it was not until in the mid-1980s that connectionist perspectives became popular in the study of human cognition and language, after the publications of the two volumes regarding the Parallel Distributed Processing (PDP) framework (McClelland, Rumelhart, & PDP research group, 1986; Rumelhart, McClelland, & PDP research group, 1986). Today connectionism has become a powerful tool as well as a conceptual framework for researchers to understand many important issues in human cognition and language.
Particularly, connectionism emphasizes 'brain-style computation,' in that connectionist networks process information in the way similar to that of the real brain, albeit in a simplified form. The human brain is a huge network made up of billions of nerve cells (neurons) and trillions of connections among them. A neuron has dendrites, to receive signals from other neurons, and axons, to send spikes to other neurons in the format of action potentials. Depending on the amount of signals it receives, a single neuron could be either ON (firing) or OFF (not firing), and it is the rate of firing of the individual neurons that defines their neural activities. Neurons are "connected" through synapses, which are actually tiny gaps with chemicals, called neurotransmitters, passing through to transmit signals. Different synapses often have different levels of strengths and effectiveness of signal transmission between neurons. Metaphorically, they are like doors between neurons, which could be widely open, open to certain extent, or even closed, with no transmitting of information occurring through the synapse. At the individual neuronal level, the electro-chemical processes for information transmission are relatively simple, going from cell bodies to axons and passing through synapses to reach the dendrites of another neuron, involving action potential propagation and neurotransmitter trafficking along the chain. It is at the network level that information processing becomes more interesting, given that a single neuron is usually connected to thousands of other neurons. The current combination of the synaptic strengths determines the neuronal teamwork, though the synaptic strengths are not fixed and can change dynamically in response to their learning environment. The ability of the human brain to derive the 'optimal' strengths for a neural network in solving any given problem is the basis of neural information processing that has inspired connectionist theories of learning, memory, and language.
With these considerations of brain features, connectionists can build artificial neural networks with two fundamental components: simple processing elements (often labeled by synonyms like units, nodes, or artificial neurons), and connections among these processing elements. Like real neurons, a node receives input from other units and can send output to other nodes. The input signals are accumulated and further transformed via a mathematical function to determine the activation value of the node. A given connectionist network can have varying numbers of nodes, many of which are connected, so that activations can spread from node to node via the corresponding connections. Like real synapses, the connections can have different levels of strength (weights), which can be adjusted according to certain learning algorithms, thereby modulating the amount of activation a source node can influence a target node. In this way, the network can develop unique combinations of weights and activation patterns of nodes in representing different input patterns from the learning environment. Unlike traditional computer programs that are dedicated to specific tasks and are fixed a priori, the weights and activation patterns in most connectionist networks are allowed to adapt continuously during learning. It is these adaptive changes that make connectionist networks interesting as models of human behavior. Each individual neuron in the brain (or a node in the model) is not very powerful, but a simultaneously activated neural network makes human cognition possible and makes connectionist models powerful in simulating human cognition. It is worth noting, though, that the real brain is an extremely complicated system, and the process described above omits many biological details. The connectionist models are simulations rather than "reduplications" of human cognitive functions.
There are two basic concepts closely related to connectionist networks. A key idea is emergentism, which indicates that thoughts (e.g., percepts, concepts, semantics) emerge from parallel interactions among the computing neurons in our brain. Another key feature of connectionist networks is its "dynamic" feature, which indicates the "non-static" status of a network, in that the network's internal representation dynamically changes in response to the complexity and demands of the learning environment. Extending these two ideas into linguistics, connectionist linguistic models often embrace the philosophy that static linguistic representations (e.g., words, concepts, grammatical structures) are emergent properties, and can be dynamically acquired from the input environment (e.g., the speech data received by the learner).
3. Network Structure and Learning Rules
To build connectionist models one needs to select the architecture of the network and determine what learning algorithms to use to adjust connection weights. The learning algorithms can be classified roughly into two big groups: supervised and unsupervised learning. Supervised learning algorithms often calculate explicit error signals to adjust the weights of the connectionist networks while unsupervised learning does not rely on explicit error signals.
In linguistic research, a popular connectionist architecture with supervised learning is a network with information feeding forward through multiple layers of nodes (usually three layers corresponding to input, hidden, and output layers; see Figure 1a). The input layer receives information from input patterns (e.g., representations of acoustic features of phonemes), the output layer provides output patterns produced by the network (e.g., classifications of phonemes according to their sound), and the hidden layer forms the network's internal representations as a result of the network's learning to map input to output (e.g., the phonological similarities between phonemes like /b/ and /p/). The most influential supervised learning algorithm in psychological and cognitive studies is "backpropagation" or "backward propagation of the errors" (Rumelhart, Hinton, & Williams, 1986). According to backpropagation, each time the network is trained with an input-to-output mapping, the discrepancy (or error, δ) between the actual output (produced by the network based on the current connection weights) and the target output (provided by the researcher) is calculated, and is propagated back to the network so that the relevant connection weights can be changed relative to the amount of error. It is as if an external teacher informs the network about the error it makes and helps it to correct the error (hence supervised learning). Continuous weight adjustments in this way lead the network to fine-tune its connection weights in response to regularities in the input–output relationships. At the end of learning, the network derives a set of weight values that allows it to take on any pattern in the input and produce the desired pattern in the output.
Another important connectionist architecture based on supervised learning is the Simple Recurrent Network (SRN; Elman, 1990). A typical SRN is like a classic three-layer backpropagation (BP) network, but with a recurrent layer of context units (see Figure 1b). During training, temporally extended sequences of input patterns (e.g., a sequence of phonemes within a word) are sent to the network, and the goal or target of the network is often to predict the upcoming items in the sequences (i.e., predicting the next phoneme given the current phoneme input in the word). With the use of context units, an SRN model can keep a copy of the hidden-unit activations at a prior point in time, and then provide this copy along with the new input to the current stage of learning (hence "recurrent" connections). This method brings into the system a dynamic memory buffer and enables the connectionist networks to effectively capture the temporal order of information. Given that language unfolds in time, the SRN therefore provides a simple but powerful mechanism for identifying structural constraints in continuous streams of linguistic input. It is important to know that, other than predicting the upcoming items in a sequence, an SRN can also be trained to associate different linguistic aspects through the network's input-output pairs (e.g., a sentence and the scene/situation that it describes, Frank, Haselager, & van Rooij, 2009; see next two sections for more on linguistic models based on SRN).
Different from supervised learning models, unsupervised learning models use no explicit error signal at the output level to adjust the weights. There have been several different types of unsupervised learning algorithms developed (see Hinton & Sejnowski, 1999). Among them, a popular type is the self-organizing map (or SOM; Kohonen, 2001).
The SOM usually consists of a two-dimensional topographic map for the organization of input representations, where each node is a unit on the map that receives input via the input-to-map connections (see Figure 2). At each training step of SOM, an input pattern (e.g., the phonological or semantic information of a word) is randomly picked out and presented to the network, which activates many units on the map, initially randomly. The SOM algorithm starts out by identifying all the incoming connection weights to each and every unit on the map, and for each unit, compares the combination of weights (called the "weight vector") with the combination of values in the input pattern (the "input vector"). If the unit's weight vector and the input vector are similar or identical by chance, the unit will receive the highest activation and is declared the best matching unit (BMU). Once a unit becomes highly active for a given input, its weight vector and that of units in its neighborhood are adjusted, such that they become more similar to the input and hence will respond to the same or similar inputs more strongly the next time. The size of the neighborhood of a BMU usually decreases as learning progresses based either on a predetermined time table (i.e., standard SOM) or on a specific function of learning outcome (i.e., the self-adjusted neighborhood function introduced in the DevLex-II model by Li, Zhao, & MacWhinney, 2007). This process continues until all the input patterns elicit specific response units in the map. As a result of this self-organizing process, the statistical structure implicit in the input is captured by the topographic structure of the SOM and can be visualized on a 2-D map as meaningful clusters. This topographic structure is a salient feature of SOM-based models since several areas in the cortex (e.g., the primary motor and somatosensory areas) are known to form topographic maps, which means that nearby locations in the brain represent adjacent areas on the body when stimulated.
Unsupervised learning may also be realized in the brain, as captured by Hebbian learning, which is clearly related to long-term potentiation (LTP) in biological systems and emphasizes the principle that "neurons that fire together wire together" (Hebb, 1949). The Hebbian learning rule can be expressed simply as ∆wkl = β .αk. αl, where β is a constant learning rate, and ∆wkl refers to change of weights from input k to l and αk and αl the associated activations of neurons k and l. The equation indicates that the connection strengths between neurons k and l will be increased as a function of their concurrent activities. Although Hebbian learning is not part of the SOM model, different self-organizing maps can be linked via adaptive connections trained by the Hebbian learning rule to simulate complex cognitive or linguistic behaviors.
In the following two sections, I would like to give a birds'-eye view of several connectionist models of various linguistic aspects in both monolingual and bilingual settings. Detailed discussions will be given to a couple of classic models influential in the field.
4. Connectionist Monolingual Models
4.1 Models of Word Processing
Many connectionist models have been applied to account for language processing in one language. McClelland and Rumelhart (1981) introduced an Interactive Activation (IA) model of visual word perception. In their model, there are three levels of nodes: (1) features of a letter such as curves, straight lines, or crossbars; (2) individual letters; and (3) words. Information at all three levels can interact with each other during word recognition, in both "bottom-up" (features to letters to words) and "top-down" (words to letters to features) fashions. Within levels, nodes compete for activation (thus inhibiting each other); across levels, nodes either inhibit or excite each other. These inhibitory and excitatory connections give rise to the appropriate activation of patterns that capture the word recognition process. The IA model was able to account for several empirical findings, such as the Word Superiority effect. But the weights in this model are fixed and thus lack connectionist learning mechanisms. Extending features of the IA model to speech, McClelland and Elman (1986) developed the TRACE model, which was the first large-scale interactive activation model of speech perception that provided a framework for investigating the dynamic interactions across levels of acoustic features, phonemes, and words as auditory signals unfold in the linguistic context. Since these early models, a large number of connectionist models emerged to simulate both normal and impaired reading of English words. For example, in 1989, Seidenberg and McClelland introduced the Triangle Model of word learning, which has the orthography of words mapped to both the phonology and the meaning of words (hence the triangle) via layers of hidden units. A detailed discussion of connectionist models of language processing can be found in Rohde and Plaut (2003).
4.2 Models of Learning Morphology
Morphology learning has received special treatment in connectionism since Rumelhart and McClelland's pioneering model in 1986 on the acquisition of the English past tense. Empirical studies have shown that young children tend to follow a "U-shape" trajectory in learning the irregular past tenses in English: at first, they seem to have mastered the correct (but a few) inflectional forms of verbs (e.g., went, broke); at the second stage, they start to produce errors that were not seen at an earlier stage (e.g., saying goed as the past tense of go, or breaked as the past tense of break); finally, children recover from these errors, along with the correct use of other regular and irregular past tenses. This type of U-shaped phenomenon has been traditionally interpreted by investigators as indicating two totally separate mechanisms underlying children's acquisition of verb past tenses: one dictionary-like rote mapping for irregular verbs, and the other the explicit representation of a rule of adding suffix -ed to regular verbs (Pinker & Price, 1988). According to this "dual mechanisms" theory, at first, children learn past tenses only by rote memory, hence the correct use of a few irregular forms such as went and broke. The errors at the second stage represent the child's discovery of a general rule controlling the formation of English regular past tenses; at this point they apply this rule over-generally to any new verb, including irregular verbs, thus leading to the "over-generalized" errors. In the end, children realize that there are rules and exceptions to the rule, finally acquiring both the regular and irregular forms by using the dual mechanisms.
While this dual-mechanism hypothesis seemed to account for the empirical data, Rumelhart and McClelland (1986) introduced a simple two layer feed-forward connectionist model (without hidden layer) to simulate this U-shaped pattern of acquisition. The R&M model was a pattern associator that makes the strong connection between a verb's stem and its phonological form of past tense, given the input of the verb stem. There were no dual mechanisms built into the model, only statistical associations for input (verb stems)—output (past tenses of verbs) relationships. Yet the R&M model was able to show the "U-shaped" patterns of learning, based on a single mechanism with no explicit rules. The R&M model provided a new way of thinking about linguistic knowledge, that is, that the notion of a rule is not a necessary component of language representation, only a useful way of language description. In 1991, Plunkett and Marchman introduced a three layer (with a hidden layer) feed-forward network trained by a back-propagation algorithm to simulate the past tense problem. In this network, the input layer receives input patterns representing the phonological information of verb stems, the output layer provides output patterns representing phonological information of past tenses, and the hidden layer forms the network's internal representations as a result of the network's learning. Using this model, they were able to identify the "U-shaped" trajectory and simulate over-generalization errors produced by children. More important, the model captured the essential role of type versus token frequencies of verbs in the learning of English past tenses, avoiding many of the problems of the R&M model criticized by some researchers (e.g., staged input for regular vs. irregular forms, which was unrealistic in child learning; see Pinker & Price, 1988).
Several other connectionist models have also been developed to simulate the learning of other types of grammatical morphology, for example, plural formation of nouns (Plunkett & Juola, 1999), grammatical and lexical aspect (Zhao & Li, 2009) and reversive prefixes of verbs (Li & MacWhinney, 1996). These models all demonstrate that single mechanisms, such as those embodied in connectionist learning, can account for the acquisition of complex grammatical structures, without recourse to the existence of separate mechanisms of linguistic rules (e.g., regular past tense formation) versus associative exceptions (e.g., irregular past tense mappings) as suggested by Pinker and colleagues.
4.3 Models of Learning Syntax
Learning of syntax holds a special status in linguistics, as syntactic structures (e.g., hierarchical recursive structures) have formed the backbone of modern linguistic theories (e.g., generative linguistics). Many connectionist researchers believe that linguistic representations could emerge from the processing of linguistic input in ways that are learned in neural networks (see the R&M model above on linguistic rules). However, it has been challenging to connectionists to demonstrate that their models can learn syntactic structures. To address this issue, Elman (1993) has successfully applied the Simple Recurrent Network (SRN) to learn the hierarchical recursive structure of English sentences. In his simulations, he would provide simple sentences first to the SRN (e.g., the mice quickly ran away), followed by complex ones (the mice the cat chases quickly ran away), and then the most complex structures (the mice the cat the dog bites chases quickly ran away). The model's success relied on 1) the recurrent architecture in which temporal order information can be recorded (see section 3. "Network Structure and Learning Rules" above for details), and 2) the 'starting small' principle, in which the network was trained on incrementally more and more complex sentences, with hierarchical recursive structures seen in relative clauses with center embedding. These characteristics of the model show that connectionist models can develop temporal representations based on mechanisms similar to human working memory, and at the same time can simulate stages of syntactic acquisition by separating the contributions of memory and language in realistic learning.
Another important aspect related to learning syntax is the famous "systematicity" debate. Fodor and Pylyshyn (1988) have argued that connectionism cannot serve as a valid architecture of human cognition because connectionist networks lack "systematicity," which is claimed to be a fundamental feature of the human mind and refers to our ability to apply correct generalizations from a limited number of seen examples to possibly infinite unseen instances. Since the start of the debate, many attempts have been made to demonstrate that connectionist linguistic models can show systematicity. For example, Frank, Haselager, & van Rooij (2009) applied an SRN to learn to comprehend sentences. Specifically, they created a "microlanguage" and a "microworld," and trained the network to map the sentences from the microlanguage (as the input) to the corresponding events from the microworld (as the output target). The authors then systematically tested their trained network with novel sentences describing new situations. The results showed that the network had learned the syntax of the microlanguage and could generalize the knowledge to comprehend new sentence-event pairs previously unseen by it (a sign of systematicity). Similarly, Farkaš and Crocker (2008) also demonstrated systematicity in a connectionist model of sentence processing, based on a self-organizing map equipped with a recurrent component. These simulation results have weakened Fodor and Pylyshyn's criticism of lacking systematicity in connectionism.
4.4 Models of Learning Semantic Structures
The learning of semantic and lexical categories or structures in the mental lexicon is another important research topic in connectionism, closely related to the learning of syntactic structures. Elman (1990) successfully used an SRN to capture semantic categories like nouns, verbs, and adjectives as language unfolds in time. Particularly, he applied hierarchical clustering analyses on the activation patterns of units in the hidden layer and demonstrated the emergence of semantic categories in the internal representation of the network. Another classic work was from Ritter and Kohonen (1989), which showed how self-organizing maps can extract topographically structured semantic categories from the linguistic input. The authors tested a single SOM, with inputs representing the meaning of words that were generated from two methods. The first method is a feature-based method, according to which a word's meaning is represented by a vector and each dimension of this vector represents a possible descriptive feature or attribute of the concept. The value of the dimension could be zero or one, indicating the absence (0) or presence (1) of a particular feature for the target word. For example, the representations of dove and hen are very similar, except one dimension represents the flying feature (dove = 1, hen = 0). Specifically, Ritter and Kohonen generated a detailed representation of 16 animals based on 13 attributes, trained a SOM with the 16 animal words, and found that the network was able to form topographically organized representations of semantic categories associated with the 16 animal words. Their second method of representing meanings of words is a statistics-based method, according to which the researchers generated a corpus consisting of three-word sentences randomly formed from a list of nouns, verbs, and adverbs (e.g., Dog drinks fast). A trigram window is applied to the corpus, and the co-occurrence frequencies of the word in the middle of the trigram with its two closest neighbors are calculated. This generates a co-occurrence matrix, which forms the basis of each word's "average context vector," a combination of the average of all the words preceding the target word and that of all the words following it. The researchers then used these vectors as input to the SOM, and training on the SOM again indicated topographically structured semantic and grammatical categories on the map.
4.5 Models of Lexical Development
Children's early lexical development is another lively debated area in language development. Several connectionist models have been developed to simulate the phenomenon of vocabulary spurt, the suddenly acceleration of word learning when a child is about 18–22 months old. Plunkett, Sinha, Møller, and Strandsby (1992) presented a multi-layered auto-associative network that showed vocabulary spurt in both comprehension and production. Regier (2005) provided a connectionist model that accounted for the occurrence of rapid vocabulary learning by highlighting children's increased ability of selective attention as vocabulary increases.
A number of connectionist models of lexical development have also been based on SOM. Specifically, Li and colleagues have introduced a series of DevLex (Developmental Lexicon) models, which have provided a SOM-based developmental framework of language acquisition in accounting for patterns of early lexical development (Li, Farkaš, & MacWhinney, 2004; Li, Zhao, & MacWhinney, 2007). The original DevLex model was introduced in 2004, and it includes two growing SOMs, connected by links trained through Hebbian learning. The model was able to capture emergence of semantic categories and age of acquisition effects in incremental lexical learning. The newest development of DevLex models is DevLex-II (Li, Zhao, & MacWhinney, 2007), which successfully simulates patterns of early lexical development, including vocabulary spurt. As seen in Figure 3, DevLex-II includes three basic levels for the representation and organization of linguistic information: phonological content, semantic content, and the output sequence of the lexicon. The core of the model is an SOM that handles lexical-semantic representation. This SOM is connected to two other SOMs, one for input (auditory) phonology, and another for articulatory sequences of output phonology. Upon training of the network, the semantic representation, input phonology, and output phonemic sequence of a word are simultaneously presented to and processed by the network. This process can be analogous to a child's analysis of a word's semantic, phonological, and phonemic information upon hearing a word. The associative connections between maps are trained via the Hebbian learning rule, which states that the weights of the associative connections between co-activated nodes on two maps will become increasingly stronger as learning and training progress (i.e., if they are frequently co-activated). The interconnected SOMs, along with the Hebbian associative weights, allow the researchers to simulate word comprehension (the activation of a word meaning via links from the phonological SOM to the semantic SOM) and word production (the activation of phonetic sequences via links from the semantic SOM to output sequence SOM).
Trained with inputs derived from a real linguistic environment (CHILDES parental corpus; MacWhinney, 2000), the DevLex II model has been able to account for a wide variety of patterns in early lexical development, including vocabulary spurt. Specifically, in DevLex-II, developmental changes such as the vocabulary spurt are modulated by lexical organization along with input characteristics. The onset of the vocabulary spurt is triggered by structured representations in the semantic, phonological, and phonemic organizations. Once these patterns have consolidated, the associative connections between maps can be reliably strengthened through learning to capture the systematic structural relationships between forms and meanings. At this point, word learning is no longer hampered by uncertainty and confusion on the maps, and the vocabulary spurt occurs. This approach is highly consistent with recent perspectives in developmental psychology and cognitive neuroscience, according to which early learning has a cascading effect on later development and learning itself shapes the cognitive and neural structures.
5. Connectionist Bilingual Models
In contrast to the flourish of research in connectionist modeling of one language, there have been only a handful of neural network models designed specifically to account for bilingualism, although this number is rapidly growing as can be seen in a recent special issue of Bilingualism: Language and Cognition, edited by Li (2013). In this section, I will briefly review a few key connectionist models in bilingual processing, bilingual development, and the interactions of bilinguals' two languages.
5.1 Models of Bilingual Processing
One of the best-known computational models of bilingualism is the Bilingual Interactive Activation (BIA) model (Dijkstra & van Heuven, 1998). The BIA model is a localist model based on the Interactive Activation (IA) model that we discussed in the last section. Incorporating the design features of the IA model into the study of bilingualism, the BIA model consists of four levels of nodes: features, letters, words, and languages. Special to the BIA model, apart from the modeling of two different lexicons, are the pre-determined language nodes (e.g., one for English and one for Dutch). Language nodes in BIA function as an important mechanism for the selection or inhibition of words in one or the other language. BIA argues for and implements the language-independent access hypothesis, according to which words from different languages are simultaneously activated during word recognition. The BIA model was later expanded as the BIA+ model (Dijkstra & van Heuven, 2002), in which nonlinguistic properties such as decision and task schemas were added to the original BIA model to account for a variety of empirical patterns. It is worth noting that, although the BIA models have found a great deal of empirical evidence in bilingual processing, they do not incorporate dynamic learning mechanisms to allow for effects of developmental changes because they were designed to capture proficient bilingual speakers' mental representation.
Based on the famous Triangle model (Seidenberg & McClelland, 1989), Filippi, Karaminis, and Thomas (2014) presented a connectionist model of language switching in bilingual production. Their work was based on simulations of behavioral experiments in which Italian-English adult bilinguals were asked to name visually presented words in the two languages (and their language switching costs were measured). The results from both their modeling and experimentation demonstrated a pattern of asymmetry: bilinguals showed a greater cost in reaction time when naming involved switching from their less fluent language to their dominant language than switching in the reverse direction. The authors argued that this switching asymmetry arises from the competition between the bilingual's two languages during production and the resolution of this competition.
5.2 Models of Bilingual Development
Recently, developmental issues in bilingualism (such as second language development and the age of acquisition (AoA) effects) have attracted the attention of many connectionists. Hereafter, I will discuss some connectionist bilingual development models.
Thomas (1997) used a Bilingual Single Network (BSN) model to learn the orthography-to-semantics mapping in visual word recognition. The BSN used a standard three-layer network with the back-propagation algorithm to transform a word's orthography (input) to a word's semantic representation (output). It was trained on vocabulary sets from two simplified artificial languages, and the network was exposed to both materials, either in a balanced condition (equal amount of training) or an unbalanced condition (L1 trained three times as often as L2). The language membership of these pseudo words was explicitly marked in both the input and output layer of the network. After learning, the network's internal representation (the activation pattern of the hidden units) of each pseudo word was analyzed by a Principal Component Analysis (PCA). This analysis indicated that under both conditions, the network was able to develop distinct internal representations for L1 vs. L2, although in the unbalanced condition the L2 words were less clearly represented, compared with those in the balanced condition.
Incorporating sentence-level input, French (1998) tested a Bilingual SRN (BSRN) model that was also trained on artificially generated sentences of the N-V-N structure in two artificial languages. The network was exposed to bilingual input as in the BSN model, but with the two artificial languages intermixed at the sentence rather than the word level, and with the input having a certain probability of switching from one language to the other. In addition, no explicit language membership marker was included in the model (i.e., no "language nodes" as in the BIA model). The model adopted the architecture of SRN model, and the task was to predict the next word given the current word input in the sentence. Simulations with the BSRN model showed that distinct patterns of the two target languages emerged after training: words from the two languages became separated in the network's internal representations (the hidden-nodes activations) as analyzed by PCA. The model provided support to the hypothesis that the bilingual input environment itself (mixed bilingual sentences in this case) is a sufficient condition for the development of a distinct mental representation of each language, without invoking separate processing or storage mechanisms for the different languages.
The BSN and BSRN models were based on supervised learning algorithms. To explore unsupervised learning for bilingualism, Li and Farkaš (2002) proposed a self-organizing model of bilingual processing (SOMBIP), in which training data derived from actual linguistic corpus were used for the model. Through Hebbian learning, the SOMBIP model connects two SOM maps: one trained on phonological representations, and the other on semantic representations. The phonological representations of words were based on articulatory features of phonemes, whereas the semantic representations were derived from the extraction of co-occurrence statistics in child-directed, bilingual, parental speech. Chinese and English were the two target languages, and the activation patterns (the distribution of BMUs corresponding to the words) on the two SOMs were analyzed. Simulation results from SOMBIP indicated that the simultaneous learning of Chinese and English led to distinct lexical representations for the two languages, as well as structured semantic and phonological representations within each language. Consistent with the general patterns of BSN and BSRN, these results suggest that natural bilingual input contains sufficient information for the learner to differentiate the two languages. An interesting aspect is that SOMBIP provides a different way to assess proficiency. By having the network exposed to fewer sentences in L2, the model simulates a novice learner having limited linguistic experience, and this differs from a pre-determined, more artificial training schedule in simulating learner differences (e.g., BSN's balanced vs. unbalanced training schedule, in which L1 was trained three times as often as L2 for the latter). This more natural way of modeling proficiency, interestingly, yielded comparable results to those from the unbalanced BSN: the 'novice' network's representation of the L2 was more compressed and less clearly delineated, compared to the 'proficient' network. Specifically, compared with L1 words, the L2 words occupied only a small portion of the semantic map, and many confusions and overlaps of BMUs occurred within the L2 area.
Zhao and Li (2010) extended their DevLex-II model to study the development of bilingual's lexical representation under different age of acquisition (AoA) of L2. The model was trained to learn 1,000 English and Chinese words, with different onset times for the learning of the second language (L2, either English or Chinese) relative to that of the first language (L1, either Chinese or English). There were three learning scenarios: simultaneous learning of L1 and L2; early L2 learning; and late L2 learning. For simultaneous learning, the two lexicons were presented to the network and trained in parallel. For early L2 learning, the onset time of L2 input to the model was slightly delayed relative to that of L1 input. For late L2 learning, the onset time of L2 input was significantly delayed relative to that of L1. Specifically, the simultaneous learning situation is analogous to a situation in which children are raised in a bilingual family and receive linguistic inputs from the two languages simultaneously. The early learning situation could be compared to the situation in which bilinguals acquire their L2 early in life (e.g., in early childhood), while the late learning situation compares to that of a bilingual's learning of L2 later in life (e.g., after puberty). The modeling results indicate that, when the learning of L2 is early relative to that of L1, functionally distinct lexical representations may be established for both languages. However, when the learning of L2 is significantly delayed relative to that of L1, the structural consolidation of the L1 lexicon becomes a hindrance to the L2 for establishing its distinct and independent lexical representation. These findings from DevLex-II provide a computational account of the age of acquisition effect by referencing the dynamic interaction and competition between the two languages competing in the bilingual mind. The reduced plasticity in L2 learning could be seen as the result of structural changes due to the learning system's experience with L1, in that L1 consolidation adversely impacts the system's ability to reorganize and restructure L2 relative to L1.
DevLex-II was also extended to simulate cross-language semantic priming in connection with the age of acquisition effect (Zhao & Li, 2013). Cross-language priming has been a vital empirical method in the literature for testing semantic representations in bilinguals, and many studies have shown that in such a paradigm bilinguals respond faster to translation equivalents or semantically related words across languages than to unrelated pairs of words from the two languages (named translation priming and semantic priming, respectively). In addition, the translation priming effects are always larger than the semantic priming. It has also been observed that priming effects are stronger if participants are presented with L1 words as primes and L2 words as targets (i.e., the L1-to-L2 direction of priming), as compared with the situation in which participants are presented with L2 words as primes and L1 words as targets (i.e., the L2-to-L1 direction of priming). More interestingly, such "priming asymmetry" decreased as a function of the effect of age of acquisition; for example, it was larger in the late L2 learning situation than in the early L2 learning situation (see Dimitropoulou, Duñabeitia, & Carreiras, 2011 for a review). Zhao and Li implemented a spreading activation mechanism in DevLex-II. As a result, the model clearly displayed cross-language priming patterns consistent with the empirical literature mentioned above: such that the stronger translation priming effects compared with semantic priming, and the "priming asymmetry".
6. Critical Issues
Voices against connectionism as a valid model of human language and learning have been there since its resurgence in the 1980's. As the result, some critical issues for neural network modeling have been identified by connectionists, and special attentions have been put on them. In the following, I will introduce a couple of them.
6.1 Representing Linguistic Features
Some early connectionist models were criticized as "toy models" that lacked linguistic and psychological reality. Critics raised the issue of whether results from such models could make direct contact with the statistical properties of natural linguistic input to which the learner or language user was exposed. Connectionist language researchers have taken this issue seriously. Many of them agree that "input representativeness" is crucial for computational modeling of language (Christiansen & Chater, 2001), and they have been concerned with how to accurately represent various linguistic aspects in their models. A crude way to represent lexical entries is to use the so-called 'localist' representation, according to which a single, unitary processing unit in the network, randomly picked by the modeler, is assigned a numerical value to represent a linguistic item (e.g., the meaning, sound, or other linguistic property of a word).
A different way, embraced by the PDP models in general, is to represent lexical entries as distributed representations, according to which a given lexical item is represented by multiple nodes and their weighted connections, as a distributed pattern of activation of relevant micro-features. For example, methods to derive distributed semantic representations of words can be roughly classified into two groups. One is the feature-based representation, in which empirical data are often used to help generate the features describing the meaning of words (e.g., McRae, Cree, Seidenberg, & McNorgan, 2005). The other is the corpus-based representation that derives meaning of words through co-occurrence statistics in large-scale linguistic corpora. Hyperspace Analogue to Language (HAL, Burgess & Lund, 1997) and Latent Semantic Analysis (LSA, Landauer & Dumais 1997) are two widely used corpus-based semantic representation methods. Zhao, Li, and Kohonen (2011) recently developed the Contextual Self-Organizing Map, a software package that can derive corpus-based semantic representations based on word co-occurrences in multiple languages, and the method has been shown to capture unique linguistic features of different languages.
It is important that researchers choose an appropriate method to represent linguistic features based on careful evaluations of their simulation goals. The localist representation definitely has the simplicity and efficiency for modeling, given its one-to-one mapping between linguistic entities and units. However, if one wants to simulate effects like similarity-based semantic priming, the distributed representations might be a better choice because they can easily capture the similarities among concepts (see Zhao & Li, 2013 for an example).
6.2 Plasticity and Stability of the Models
Many classic connectionist models have been challenged for their plasticity (or lack of it) to learn new knowledge. When faced with a sequential learning task (such as learning a second language), they may encounter a critical issue called "plasticity-stability" dilemma (see Mermillod, Bugaiska, & Bonin, 2013). Particularly, a connectionist network needs plasticity to learn new knowledge, but too much plasticity often causes it to lose its stability for old knowledge; conversely, a network that is too stable often cannot adapt itself very well to the new learning task. Taken second language learning as an example, if we train a network to acquire an L1 lexicon with 500 words and then train it on another 500 words in L2, in many traditional networks, the addition of L2 words may disrupt the network's knowledge of L1. In other words, the network will lose its representation of L1 words because of learning new words. Such a "catastrophic interference" (French, 1999) of course, is unlike human learning. Researchers should be aware that remedies may be needed for this problem depending on their research goals. For example, some connectionist models have adjustable network structures, or involve dynamic unit growth, such as the cascade-correction network (Shultz & Fahlman, 2010), the constructivist neural network (Ruh & Westermann, 2009), and the Growing Neural Gas model (Fritzke, 1995). Our DevLex models (Li, Farkaš, & MacWhinney, 2004; Li, Zhao, & MacWhinney, 2007; Zhao & Li, 2010) also integrated certain features (like the self-adjusted neighborhood function and the growing SOMs) to resolve the plasticity-stability problem. Similar example can also be found in the interconnected growing SOMs model from Cao, Li, Fang, Kaufmann, and Kröger (2014).
6.3 Network Structure Selection and Parameters Setup
From the previous discussions, it should have become clear to readers that choosing appropriate network structures is a key to the success simulations of their specific linguistic questions. For example, if the researcher is interested in semantic representation and organization, a SOM-based architecture might be highly appropriate and relevant given its topography-preserving feature. But if the researcher is interested in simulating the processing of temporally ordered components (e.g., syntax), a network with a recurrent portion resembling the SRN structure might be a better candidate. Other connectionist network structures, such as "hybrid" models that combine both supervised and unsupervised learning methods, and models that have adjustable structures or involve dynamic unit growth (see discussion in section 6.2 "Plasticity and Stability of the Models"), may also be considered based on the researchers' simulation needs.
Regardless of the type of learning algorithm a researcher uses, every model will involve a set of "parameters" that need to be considered and set up. Among them, there are research variables whose effects on the network's output are of interest to the researchers. A benefit of computational modeling is that researchers can easily bring a research variable under tight experimental control by systematically manipulating different levels of the variable, and they can see the effects while holding other variables constant. In this way, we can investigate the outcome caused by the use of different levels of a specific research variable, which may be difficult to manipulate in the natural environment.
Other than the research variables, there are also some "free parameters" in a model that often need to be adjusted by the modelers. For example, how large should the size of the network be to adequately learn the target aspect of the second language? What value should be assigned to the learning rate? Decisions need to be made by the researchers in advance on such practical questions before a simulation is run. Often, there are no correct answers to these questions, as each model involves a different degree of complexity and task difficulty, and the researcher needs to use experience based on previous models and conventional wisdom in setting up appropriate values of the free parameters. Inevitably, criticism may arise toward such practice in a particular model. One caution here is that the researcher should avoid introducing too many free parameters in a simulation project. Although more free parameters usually mean better fitting of the model to the target data, it may compromise the external validity of the network in relating to the phenomena being simulated. As in empirical studies, findings from overly tight controlled experiments with too many variables under consideration may not generalize to other situations.
7. Theoretical Linguistics Inspired by Connectionism
In the previous sections, the usefulness of connectionist network models for language studies has been demonstrated. At this point, it should become clear that computational models, especially connectionist models, have much to offer to the understanding of linguistic behavior. Connectionism argues in favor of a computational approach to the complexities associated with the dynamical interactions of the learning system with the linguistic environment. From the many connectionist language studies of last three decades, researchers has demonstrated the biological plausibility and psychological reality of the idea that complex human linguistic behavior can emerge from simple neural-chemical processes. In this perspective, the rules of language can be considered as the product of neural network processes in the long human evolutionary history, in much the same way as a hexagonal structure emerges from the simple acts of honeybees packing small amounts of honey into the honeycomb from multiple directions (Bates, 1984).
Many influential modern linguistic theories, such as the Optimality Theory, can find their roots in connectionism. Indeed the core of the OT theory is the computation of the optimal representation with maximum harmony, which has its similarity to the search of minimum error in connectionist models (see Smolensky & Legrendre, 2006). Bybee (2001) argues that phonological patterns can be understood as emergent consequences of language usage (e.g., frequency of a word and its co-occurence with other words), and she argues against domain-specific modules in the brain for language processing. These ideas are highly consistent with connectionism. With the profound impact of connectionism in the last three decades, it has become increasingly clear that language can no longer be studied as a pure innate symbolic system, as advocated in classic works on linguistics, and it is expected that more linguistic theories and studies will be inspired by connectionism.
Further Reading
Comparison of Neural Network Simulators (2015, May 20). Emergent simulation environment.
Elman, J., Bates, A., Johnson, A., Karmiloff-Smith, A., Parisi, D., & Plunkett, K. (1996). Rethinking innateness: A connectionist perspective on development. Cambridge, MA: MIT Press.Find this resource:
Kohonen, T. (2001). Self-organizing maps (3d ed.). Berlin, Germany: Springer.Find this resource:
Li, P. (2009). Lexical organization and competition in first and second languages: Computational and neural mechanisms. Cognitive Science, 33, 629–664.Find this resource:
Li, P., & Zhao, X. (2012). Connectionism. In M. Aronoff (Ed.), Oxford Bibliographies Online: Linguistics. New York: Oxford University Press.Find this resource:
Li, P., & Zhao, X. (2013). Self-organizing map models of language acquisition. Frontiers in Psychology, 4, 828.Find this resource:
MacWhinney, B., & Li, P. (2008). Neurolinguistic computational models. In B. Stemmer & W. Whitaker (Eds.), Handbook of the neuroscience of language (pp. 229–236). London: Elsevier Science Publisher.Find this resource:
McClelland, J. (2009). The place of modeling in cognitive science. Topics in Cognitive Science, 1, 11–28.Find this resource:
McClelland, J. (2014) Explorations in parallel distributed processing: A handbook of models, programs, and exercises.
Ritter, H., & Kohonen, T. (1989). Self-organizing semantic maps. Biological Cybernetics, 61, 241–254.Find this resource:
Rumelhart, D. (1989). The architecture of mind: A connectionist approach. In M. Posner (Ed.), Foundations of cognitive science. Cambridge, MA: MIT Press.Find this resource:
Rumelhart, D., Hinton, G., & Williams, R. (1986). Learning internal representations by error propagation. In D. Rumelhart, J. McClelland, & the PDP Research Group (Eds.), Parallel distributed processing: Explorations in the microstructures of cognition (Vol. 1, pp. 318–362). Cambridge, MA: MIT Press.Find this resource:
Spencer, J. P., Thomas, M. S. C., & McClelland, J. L. (Eds.) (2009). Toward a unified theory of development: Connectionism and dynamic systems theory re-considered. Oxford: Oxford University Press.Find this resource:
Spitzer, M. (1999). The mind within the net: Models of learning, thinking, and acting. Cambridge, MA: MIT Press.Find this resource:
Westermann, G., Ruh, N., & Plunkett, K. (2009). Connectionist approaches to language learning. Linguistics, 47, 413–452.Find this resource:
Bates, E. (1984). Bioprograms and the innateness hypothesis. Behavioral and Brain Sciences, 7, 188–190.Find this resource:
Burgess, C., & Lund, K. (1997). Modeling parsing constraints with high-dimensional context space. Language and Cognitive Processes, 12, 177–210.Find this resource:
Bybee, J. (2001). Phonology and language use. Cambridge, U.K.: Cambridge University Press.Find this resource:
Cao, M., Li, A., Fang, Q., Kaufmann, E., & Kröger, B. J. (2014). Interconnected growing self-organizing maps for auditory and semantic acquisition modeling. Frontiers in Psychology, 5, 236.Find this resource:
Christiansen, M. H., & Chater, N. (2001). Connectionist psycholinguistics: Capturing the empirical data. Trends in Cognitive Sciences, 5, 82–88.Find this resource:
Dijkstra, T., & van Heuven, W. (1998). The BIA model and bilingual word recognition. In J. Grainger & A. M. Jacobs (Eds.), Localist connectionist approaches to human cognition (pp. 189–225). Mahwah, NJ: Erlbaum.Find this resource:
Dijkstra, T., & Van Heuven, W. (2002). The architecture of the bilingual word recognition system: From identification to decision. Bilingualism: Language and Cognition, 5(3), 175–197.Find this resource:
Dimitropoulou, M., Duñabeitia, J. A., & Carreiras, M. (2011). Two words, one meaning: Evidence of automatic co-activation of translation equivalents. Frontiers in Psychology, 2, 188.Find this resource:
Elman, J. (1990). Finding structure in time. Cognitive Science, 14, 179–211.Find this resource:
Elman, J. (1993). Learning and development in neural networks: The importance of starting small. Cognition, 48(1), 71–99.Find this resource:
Farkaš, I., & Crocker, M. W. (2008). Syntactic systematicity in sentence processing with a recurrent self-organizing network. Neurocomputing, 71(7), 1172–1179.Find this resource:
Filippi, R., Karaminis, T., & Thomas, M. (2014). Language switching in bilingual production: Empirical data and computational modelling. Bilingualism: Language and Cognition, 17, 294–315.Find this resource:
Fodor, J. A., & Pylyshyn, Z. W. (1988). Connectionism and cognitive architecture: A critical analysis. Cognition, 28, 3–71.Find this resource:
Frank, S. L., Haselager, W. F., & van Rooij, I. (2009). Connectionist semantic systematicity. Cognition, 110(3), 358–379.Find this resource:
French, R. M. (1998). A simple recurrent network model of bilingual memory. In M. A. Gernsbacher & S. J. Derry (Eds.), Proceedings of the 20th Annual Conference of the Cognitive Science Society (pp. 368–373). Mahwah, NJ: Erlbaum.Find this resource:
French, R. M. (1999). Catastrophic forgetting in connectionist networks. Trends in Cognitive Sciences, 3, 128–135.Find this resource:
Fritzke, B. (1995). A growing neural gas network learns topologies. In G. Tesauro, D. S. Touretzky, & T. K. Leen (Eds.), Advances in neural information processing systems 7: Proceedings of the 1994 conference (pp. 625–632). Cambridge, MA: MIT Press.Find this resource:
Hebb, D. (1949). The organization of behavior: A neuropsychological theory. New York: Wiley.Find this resource:
Hinton, G. E., & Sejnowski, T. J. (1999). Unsupervised learning: Foundations of neural computation. Cambridge, MA: MIT press.Find this resource:
Kohonen, T. (2001). Self-organizing maps (3d ed.). Berlin, Germany: Springer.Find this resource:
Landauer, T. K., & Dumais, S. T. (1997). A solution to Plato's problem: The latent semantic analysis theory of the acquisition, induction, and representation of knowledge. Psychological Review, 104, 211–240.Find this resource:
Li, P. (2013). Computational modeling of bilingualism. [Special issue]. Bilingualism: Language and Cognition, 16(2), 241–366.Find this resource:
Li, P., & Farkaš, I. (2002). A self-organizing connectionist model of bilingual processing. In R. Heredia & J. Altarriba (Eds.), Bilingual sentence processing (pp. 59–85). North-Holland, Netherlands: Elsevier Science Publisher.Find this resource:
Li, P., Farkaš, I., & MacWhinney, B. (2004). Early lexical development in a self-organizing neural network. Neural Networks, 17, 1345–1362.Find this resource:
Li, P., & MacWhinney, B. (1996). Cryptotype, overgeneralization, and competition: A connectionist model of the learning of English reversive prefixes. Connection Science, 8(1), 3–30.Find this resource:
Li, P, Zhao, X., & MacWhinney, B. (2007). Dynamic self-organization and early lexical development in children. Cognitive Science: A Multidisciplinary Journal, 31, 581–612.Find this resource:
MacWhinney, B. (2000). The CHILDES project: Tools for analyzing talk, transcription, format and programs (Vol. 1). Mahwah, NJ: Lawrence Erlbaum.Find this resource:
McClelland, J., & Rumelhart, D. (1981). An interactive activation model of context effects in letter perception: Part 1. An account of basic findings. Psychological Review, 88, 375–407.Find this resource:
McClelland, J., Rumelhart, D., & the PDP Research Group (1986). Parallel distributed processing: Explorations in the microstructure of cognition (Vol. 2). Cambridge, MA: MIT Press.Find this resource:
McClelland, J. L. (2009). The place of modeling in cognitive science. Topics in Cognitive Science, 1, 11–28.Find this resource:
McClelland, J. L., & Elman, J. (1986). The TRACE model of speech perception. Cognitive Psychology, 18, 1–86.Find this resource:
McRae, K., Cree, G. S., Seidenberg, M. S., & McNorgan, C. (2005). Semantic feature production norms for a large set of living and nonliving things. Behavior Research Methods, 37(4), 547–559.Find this resource:
Mermillod, M., Bugaiska, A., & Bonin, P. (2013). The stability-plasticity dilemma: Investigating the continuum from catastrophic forgetting to age-limited learning effects. Frontiers in Psychology, 4, 504.Find this resource:
Pinker, S., & Prince, A. (1988). On language and connectionism: Analysis of a parallel distributed processing model of language acquisition. Cognition, 28, 73–193.Find this resource:
Plunkett, K., & Juola, P. (1999). A connectionist model of English past tense and plural morphology. Cognitive Science, 23(4), 463–490.Find this resource:
Plunkett, K., & Marchman, V. (1991). U-shaped learning and frequency effects in a multi-layered perceptron: Implications for child language acquisition. Cognition, 38, 43–102.Find this resource:
Plunkett, K., Sinha, C., Møller, M. F., & Strandsby, O. (1992). Symbol grounding or the emergence of symbols? Vocabulary growth in children and a connectionist net. Connection Science, 4(3–4), 293–312.Find this resource:
Regier, T. (2005). The emergence of words: Attentional learning in form and meaning. Cognitive Science, 29(6), 819–865.Find this resource:
Ritter, H., & Kohonen, T. (1989). Self-organizing semantic maps. Biological Cybernetics, 61, 241–254.Find this resource:
Rohde, D. L. T., & Plaut, D. C. (2003). Connectionist models of language processing. Cognitive Studies, 10, 10–28.Find this resource:
Ruh, N., & Westermann, G. (2009) Simulating German verb inflection with a constructivist neural network. In J. Mayor, N. Ruh, & K. Plunkett (Eds.), Connectionist models of behavior and cognition (Vol. 2, pp. 313–324). London: World Scientific.Find this resource:
Rumelhart, D., Hinton, G., & Williams, R. (1986). Learning internal representations by error propagation. In D. Rumelhart, J. McClelland, & the PDP Research Group (Eds.), Parallel distributed processing: Explorations in the microstructures of cognition (Vol. 1, pp. 318–362). Cambridge, MA: MIT Press.Find this resource:
Rumelhart, D., & McClelland, J. L. (1986). On learning the past tenses of English verbs. In J. L. McClelland, D. E. Rumelhart, & the PDP Research Group (Eds.), Parallel distributed processing: Explorations in the microstructure of cognition (Vol. 2), Psychological and biological models (pp. 216–271). Cambridge, MA: MIT Press.Find this resource:
Rumelhart, D. E., McClelland, J. L., & the PDP Research Group (Eds.) (1986). Parallel distributed processing: Explorations in the microstructure of cognition (Vol. 1), Foundations. Cambridge, MA: MIT Press.Find this resource:
Seidenberg, M. S., & McClelland, J. L. (1989). A distributed developmental model of word recognition and naming. Psychological Review, 96, 523–568.Find this resource:
Shultz, T. R., & Fahlman, S. E. (2010). Cascade-correlation. In C. Sammut, G. I. Webb (Eds.), Encyclopedia of machine learning (Part 4/C, 139–147). Heidelberg, Germany: Springer-Verlag.Find this resource:
Smolensky, P., & Legrendre, G. (Eds.) (2006). The harmonic mind: From neural computation to optimality-theoretic grammar (2 vols.). Cambridge, MA: MIT Press.Find this resource:
Thomas, M. S. C. (1997). Connectionist networks and knowledge representation: The case of bilingual lexical processing. Doctoral diss., Oxford University.Find this resource:
Zhao, X., & Li, P. (2009). The acquisition of lexical and grammatical aspect in a developmental lexicon model. Linguistics, 47, 1075–1112.Find this resource:
Zhao, X., & Li, P. (2010). Bilingual lexical interactions in an unsupervised neural network model. International Journal of Bilingual Education and Bilingualism, 13, 505–524.Find this resource:
Zhao, X., & Li, P. (2013). Simulating cross-language priming with a dynamic computational model of the lexicon. Bilingualism: Language and Cognition, 16, 288–303.Find this resource:
Zhao, X., Li, P., & Kohonen, T. (2011). Contextual self-organizing map: Software for constructing semantic representation. Behavior Research Methods, 43, 77–88.Find this resource:


text:
Don't tuck sun protection away with your swimwear this fall! There may not be a lot of your body for the sun to reach in the fall and winter, but if your face is exposed, you're still vulnerable. Powerful UVA rays are around all year and can cause significant damage to your skin.
Apply a moisturizer with SPF 30 to your face as part of your daily skincare routine; couple that with a daily dose of an antioxidant supplement like Heliocare® ($29.99; Walgreens). An extract from a well-studied tropical fern called Polypodium leucotomos, Heliocare®has been shown to help maintain the skin's ability to protect against sun-related effects and aging.*
Polypodium leucotomos has been evaluated for over 30 years by scientists and dermatologists in several countries, including the US. It harnesses the natural and powerful antioxidants in Polypodium leucotomos extract, which aid in eliminating free radicals produced by sun exposure. A reduction in these free radicals helps to maintain younger, more resilient skin.*
* These statements have not been evaluated by the Food and Drug Administration. This product is not intended to diagnose, treat, cure or prevent any disease.
Published on Dec 27, 2013

subset: commoncrawl


text:
Habs bring in Budaj for two years
Montreal, QC – The Montreal Canadiens signed goaltender Peter Budaj to a two-year contract extension on Wednesday.
Financial terms were not disclosed.
Budaj, 30, has played eight games with seven starts for the Canadiens this season. He has posted a 6-1-1 record with one shutout, a 2.17 goals against average and .917 save percentage.
The eight-year veteran has a 112-99-33 career record with a 2.79 GAA and a .903 save percentage in 267 games with Colorado (2005-11) and Montreal (2011-13).


text:
MOST people dont think I am a prick.Only you,because you ran out of pathetic excuses. and a couple of total idiots that can only spew abusive shit. A lot of people support me. It saves them having to post about your stupid crap.
The point is,you dont know jackshit about Cuben fibre,or anything else for that matter. Just like you knew jackshit about the demo kites.
Thanks to the guys that DO know that replied. Interesting stuff.
You even had to do a google to get info on Rick !
Ask Gumball where Rick fits in. He knows. You can tell him to stop plaguerising.


text:
Gainesville FL Real Estate
Nestled in the heart of beautiful North Central Florida, Gainesville has the
best of everything the Sunshine State has to offer. A mild climate, an active
cultural community, a healthy economy and an abundance of natural beauty are
just a few of the reasons Gainesville has been consistently recognized as one of
Florida's most livable cities.
A wide variety of home types and styles in Gainesville and Alachua County offer
something for everyone. Single family homes for professionals and families,
apartments, and condominiums with generous amenities are just a few of the
housing choices available. Several retirement communities are in the county,
which has often been ranked as one of the leading retirement areas in the nation
Gainesville's latest honor is to have been named 47 th in
CNNmoney.com's "Top 100 Places to Live and Launch" for 2008. Gainesville
was also voted best place to live in America in the "2007 Cities Ranked and
Rated" .
Wildlife areas in and around Alachua County are plentiful as are lakes, rivers
and springs. In addition to fishing, hiking and biking, the favorable weather
makes golf, tennis, running and other outdoor pursuits popular with residents as
well. Areas of interest are Bivens Arm Nature Park, Devil's Millhopper,
Gainesville-Hawthorne State Trail, Kanapaha Botanical Gardens, Paynes Prairie
State Preserve, and Santa Fe Community College Teaching Zoo, to name just a few.
Patrons of the arts and culture will find no shortage of things to do and see in
the area. Dance, music, theater, visual arts, and folklore are an integral part
of the community. The presence of a diverse student body and faculty at the
University of Florida contributes to the social and cultural life of
Several small nearby communities, such as High Springs, Alachua, Micanopy,
Archer, Hawthorne, Keystone Heights, Newberry and McIntosh are unusually rich in
historical sites and structures. Like any college town, the nightlife scene is
always buzzing with dance clubs, jazz clubs, and theatrical productions. Major
collegiate spectator sports are another recreational option, especially when any
sports team for the University of Florida Gators has a home game.
The Gainesville/Alachua County area is North Central Florida's employment hub.
Its diversified labor force of more than 100,000 employees has overall education
levels significantly higher than the state averages. U.F., along with government
agencies, is the largest employer in the county. Healthcare is second, with
world renowned facilities at Shands and the V.A. Hospitals, followed by retail
and service jobs, plus a growing number of high-tech, research, environmental,
engineering and agricultural interests.
Despite a downturn in the housing market in other areas nationwide, buyers still
get plenty for their money in Gainesville/Alachua County. There are great buys
in the area, but you need a local market expert to help you. Watson Realty Corp.
is the largest independently owned real estate brokerage firm in North Florida
and has been in business since 1964. Our staff of dedicated real estate
professionals is committed to making your move as simple and painless as


text:
Rheem Complete Split System Gas Furnace 2 Ton 16 SEER 13 EER. RGPE Series
Short Desc.: Outside condenser, Gas Furnace and Cased Coil
Durability for Years to Come : While blending effortlessly into your home's landscape, the attractive, low-profile, louvered, Neutra Green™ steel cabinet provides style and protection from yard hazards and weather extremes. With this durable construction, you can count on years of ease when you depend on Rheem for your home comfort needs.
Tranquility of Quiet Operation: Air is discharged through a unique grille pattern located on top of the unit, and is directed upward and away from bushes and shrubs. This discharge pattern allows for quiet fan operation and low noise levels, which you and your neighbors will appreciate.
Simple, Low Cost System Maintenance: Contractor-preferred access, to the internal components of the unit, is easily obtained by removing one service panel or the top grille, keeping service time and expense to an absolute minimum. This enhanced serviceability makes caring for your system as easy as 1-2-3.
Efficient Performance Maximizes Savings & Comfort: The 14AJM model features cooling efficiencies up to 16.50 SEER/14 EER. The higher the SEER level, the greater your energy savings will be over time. When compared to lower SEER rated models, choosing a higher efficiency model will pay off with years of savings on your monthly energy bills.
Guaranteed Dependability: The 14.5 SEER model features a ten-year conditional limited compressor warranty, while all other parts have a five-year limited warranty (registration required). You can relax and simply enjoy the peace of mind you deserve with the worry-free operation of your Rheem Condensing Unit.
Eco-Friendly Product: At Rheem, green is much more than the patented Neutra Green™ color of our products. Our eco-friendly condensing units meet or exceed world environmental standards by utilizing Refrigerant 410A that does not contribute to ozone depletion in any way.
Consumer Literature Sales Fact Sheets Specification Sheets Submittal Sheets Tax Credit Form 2011
Gas Furnace Unit:
The Rheem Prestige Series™ Super Quiet 80™ Two Stage line of upflow/horizontal gas furnaces are designed for utility rooms, closets, alcoves, or attics. Because of the furnaces low-profile 34 inch [864 mm] height, the upflow model installed vertically can also be used to satisfy most applications that traditionally call for a horizontal furnace.
Consumer Literature Sales Fact Sheets Specification Sheet Submittal Sheets Tax Credit Form 2011 Use and Care Instructions
Cased Coil Unit:
Rheem®Indoor Coils are designed for use with Rheem out- door units and are available for vertical upflow or downflow, and horizontal left or horizontal right airflow. When matched with Rheem outdoor units, the coils provide a nominal capacity range from 24,000 BTU/HR [7.03 kW] to 60,000 BTU/HR [17.58 kW].
Documents: Specification Sheet


text:
Related story: Guilt phase of trial begins for Rancho Cucamonga man accused of fatally beating mom with golf club
Eric Nash was driving home with pizza for his family last year in Rancho Cucamonga, when he spotted neighbor Ezra Kang and Kang's son, Luke, in the street holding a golf club.
Ezra Kang stopped Nash, who lived in the same neighborhood, and asked him for and to call 9-1-1.
"He said his son has mental problems and had beat his wife. He said she's dead," Nash testified Thursday during Day 2 of a trial for Luke Kang in West Valley Superior Court, in Rancho Cucamonga.
Nash was the first person to take the witness stand during trial Thursday. Luke Kang, 28, is on trial and accused of beating his mother to death with a golf club on Feb. 22, 2012 at their Rancho Cucamonga townhome after a road trip to Los Angeles.
Hannah Kang, 60, was a registered nurse, who did home health care for young children, at the time that she died. Ezra Kang was a retired Christian pastor.
Nash testified in court that Luke Kang was in the street breathing hard, but doing anything else.
"They were holding on to the golf club," Nash said. "Neither (of them) wanted let go."
Nash called 9-1-1, he testified, and an operator asked him to go inside the house and check on Hannah Kang.
"What was her condition when you saw her?" Deputy District Attorney Karen who is prosecuting the case, asked Nash.
"Grisly. It was bad," Nash replied. Hannah Kang was breathing heavily and loudly.
When Schmauss played the 9-1-1 tape in court, Hannah Kang could be heard moaning the background.
"I just held her hand and told her help was on the way," Nash testified.
Mark Addy, a San Bernardino County sheriff's deputy, responded that night to home in the 11500 block of Stoneridge Drive, in Rancho Cucamonga, he said. Addy was the second witness to testify Thursday morning.
Hannah Kang had massive facial injuries, was having a hard time breathing and severely beaten, Addy testified. Luke Kang had been placed in the back of a car, he said. Addy described him as calm and cooperative.
Luke Kang has pleaded not guilty by reason of insanity to charges of murder and assault for allegedly attacking his mother and later his father.
During lawyer's opening statements Wednesday, Schmauss said the Kangs were both devoted to their son, who was taking medication for mental illness, and were very concerned about him.
More details to come.


text:
Jesus has said he will show himself to the one who loves him (v. 21), so Judas (not Judas Iscariot) asks, But, Lord, why do you intend to show yourself to us and not to the world? (v. 22). The term used for show (emphanizo) is used in the Septuagint for the theophany Moses received on Sinai (Ex 33:13, 18). Judas seems to be confused because he is "looking for another theophany that will startle the world" (Brown 1970:647), but Jesus is only speaking of showing himself to his disciples.
As is often the case, Jesus does not seem to address the question directly, yet in fact he goes to the heart of the issue. Judas has spoken of the contrast between us and the world, and Jesus describes the disciple as one who loves him (v. 23): If anyone loves me, he will obey my teaching (v. 23; or "keep my word," ton logon mou teresei). Jesus is referring not to simply holding onto his teaching, but to actually acting in accordance with it, as he himself has responded to the Father (v. 31). His teaching is not just interesting thoughts about God and the world. Rather, he has revealed God and opened the way to share God's own life. To obey his teaching is to adopt God's pattern of life. But the condition for such obedience is love for Jesus. The commands of Jesus are not a set of rules like a traffic code; they are a description of a pattern of life that reflects God's own life trans-posed into human circumstances. Love for Jesus involves both an attachment to him and a oneness with him and his interests, which naturally leads one to obey him and walk as he walked (1 Jn 2:6). One obeys what one loves. Indeed, our patterns of obedience reveal what we really love.
After describing the one to whom he will show himself, Jesus speaks of the showing itself: My Father will love him, and we will come to him and make our home with him (v. 23). Instead of describing a spectacular theophany, Jesus speaks of dwelling with his disciples. The word for home (mone) is the same used earlier of the "rooms" in the Father's house (v. 2). The future intimacy in heaven will begin already here on earth. The great prophetic hope of a time when God would dwell with his people (Ezek 37:26-27; Zech 2:10) has come to pass in the incarnation and the dwelling Jesus here mentions.
In this passage, as throughout the Gospel, we have the dependency of the Son upon the Father. The Father, in love, sent the Son, and so those who receive the Son in love will receive this love of the Father. For the word that they obey in love is not the Son's but that of the Father himself (v. 24). Jesus' word is not the word of a mere human teacher that can be debated and modified; it comes from the Father and thus is and expresses ultimate reality. Those who do not love and obey the Son reject the Father himself (v. 24; cf. 1 Jn 2:23). The opponents are not able to hear Jesus' word from the Father (8:43), but the disciples receive the Father's word through the Son and take into their lives that which is of God, thus sharing in his love. The Son does not come to the disciples on his own, but, just as with the incarnation itself, this new mode of dwelling with them will be initiated by the Father's love. The Son continues to do what he sees the Father doing, and together he and the Father come to the disciple. The divinity of the Son, his oneness with the Father, again underlies what is being said (cf. Westcott 1908:2:181).
While in the future the Father and the Son will make their dwelling with (para) the true disciple (v. 23), in the meantime Jesus is still with (para) them, giving them further words to receive and obey (v. 25). He realizes that there is no way they can understand what he has just been explaining to them, so he comforts them with the promise of an interpreter: But the Counselor, the Holy Spirit, whom the Father will send in my name, will teach you all things and will remind you of everything I have said to you (v. 26). Here is the second of the Paraclete passages (cf. v. 16). Jesus has just referred to himself as one sent by the Father (v. 24), and now he says the same of the Paraclete. This is the only place the Paraclete is identified with the Holy Spirit, which indicates that the Paraclete passages convey only part of the larger teaching about the Holy Spirit, focusing mainly on the role of witness and instruction. Earlier it was said the Paraclete is sent at Jesus' request (v. 16), and now it is said that he is sent in my name. This expression, as we have seen elsewhere (see comment on v. 13), includes the idea of union. As the disciple's prayer is to be in conformity with Jesus' character and actually in union with Jesus' own intent (v. 13), so the Paraclete himself is in union with Jesus and in conformity with his character and mission. "Jesus bore God's name (17:11, 12) because he was the revelation of God to men; the Spirit is sent in Jesus' name because he unfolds the meaning of Jesus for men" (Brown 1970:653). Thus, the Paraclete will bear witness to Jesus just as Jesus has borne witness to the Father, having come in his Name (5:43; 10:25).
Specifically, the Paraclete will teach and remind. In John, to remember something means both to recall it and understand it (see comment on 2:22; Mussner 1967). Teaching and reminding probably should not be seen as two separate activities but instead as two ways of speaking of the same thing (the kai would be epexegetic; cf. Schnackenburg 1982:83), so verse 26 is perhaps better translated as "that one will teach you everything, that is, he will remind you of everything which I said to you." The all things that the Paraclete will teach the disciples does not refer to knowledge of all sorts, such as the height of Mount Everest or the general theory of relativity. God is indeed the God of all creation, but the all things spoken of here is the revelation of himself that has come in Jesus (see comment on 16:14). The Spirit understands all about Jesus and will clarify all that he has taught (cf. 1 Cor 2:11-12). This word "all" (panta, translated all things and everything in the NIV) speaks of the comprehensiveness of the Spirit's work; he will leave out nothing of what Jesus has taught. Later we will learn that Jesus himself has left out nothing of what he has learned from the Father (15:15), and all that belongs to the Father is his (5:20; 16:15; 17:10). Thus, Jesus is the fullness of the revelation of the Father. No further revelation is needed, nor would it be possible. What is called for is an understanding of the revelation that has come in Jesus, and this is what the Paraclete will provide.
The promise that the Father and the Son will dwell with believers is in close proximity to the promise of the Spirit. This has led many to understand the presence of the Father and Son as being mediated by the Spirit (cf. Turner 1992:349-50), though others point out that the text does not say as much (Beasley-Murray 1987:258, 260). It is clear that the Father and the Son are personally present with the believers and that the Spirit has a role clearly distinguished from, though in union with, the Father and the Son. The Paraclete's teaching role is focused on the historical Jesus, as indicated by the reference to all things and everything (v. 26) and the use of the past tense (eipon, translated I have said). This focus on the Son is further emphasized by the inclusion of the emphatic personal pronoun "I" (ego, in v. 26: everything I have said to you) though the manuscripts vary at this point. Later passages will also indicate that Jesus himself continues to instruct the disciples, which suggests the Spirit mediates Jesus' presence (see comment on 16:25).
The other distinctive is the gift of peace that Jesus gives them: Peace I leave with you; my peace I give you. I do not give to you as the world gives (v. 27). Here is the fulfillment of the prophets' promise of peace (for example, Is 9:6-7; 52:7; Ezek 37:26; cf. Beasley-Murray 1987:262). The peace Jesus is talking about is not the cessation of hostilities from enemies, but rather the gift of calmness and confidence that comes from union with God and faith in him and his purposes. The world's idea of peace is something that comes through destroying of enemies and consists of physical and emotional comfort. The peace that Jesus gives is grounded in God and not in circumstances. It is the peace that Jesus himself has exhibited in this Gospel and is exhibiting in this farewell discourse, even while he knows he is about to be killed. Soon he will speak of the continued trouble his disciples will experience in the world (15:18—16:4), but they will simply be living out what he himself has already been experiencing. They will share his troubles, but they will also have his peace, for they will share in his own relationship with the Father.
This promise of the gift of his own peace serves as the foundation for the command he now gives: Do not let your hearts be troubled and do not be afraid (v. 27). He repeats exactly the command that began this chapter (v. 1), adding now a reference to fear (mede deiliato). This word family is always used of fear in a negative sense, as the opposite of courage. Those with a settled disposition of such fear evidence a lack of faith in God and a denial of his presence, his goodness and his power. Those who experience such fear, which includes virtually all of us to some degree, may take comfort that as God's life grows within us and as our hearts are healed, we enter into the inheritance of Jesus' peace, which replaces our sinful fear. Jesus here calls us to receive his peace. The grounds of this peace is the "perfect love" that "drives out fear" (1 Jn 4:18). This love is ultimately a sharing of the relationship between the Father and the Son, of which Jesus now goes on to speak.
His announcement that he is departing to the Father should fill them with joy instead of disturbance and fear (v. 28). The construction in Greek of the phrase If you loved me indicates that Jesus' view is that they have not done so. So their response shows that they have not yet come to love him in the truest sense. They think they love him, but in fact they are more focused on themselves than on him (Westcott 1908:2:185). Fear in itself is focused on self and circumstances rather than on God. Focus on God is central to all Jesus does and says, as it is here: If you loved me, you would be glad that I am going to the Father, for the Father is greater than I (v. 28). Jesus' great love and focus is the Father; thus the prospect of returning to him fills Jesus with joy. If the disciples shared this focus and really loved Jesus, that is, willed the best for him, they also would share this joy.
Jesus' statement that the Father is greater than I is very important for understanding the relation between the Father and the Son. Arius, who lived in the fourth century, and others who have held views similar to his since then have taken this verse as proof that Jesus is not divine. The teachers of the church rejected this notion, and indeed it is not compatible with other material in this very Gospel. It has been clear from the first verse that the Son is one with God yet distinct from God (especially 1:1-18; 8:58; 10:30; 20:28). In fact, this distinctness is now further clarified by Jesus' saying the Father is greater. From the time of the early church this verse has been the focus of much thought (cf. Westcott 1908:2:191-96; Pollard 1970). There have been two main ways to understand this verse that do justice to the oneness of the Father and the Son.
First, some say that the verse's focus is on Jesus' historical mission. The Father is greater in that he is the source and goal of Jesus' mission (for example, Calvin 1959:89-90; Brown 1970:655; Schnackenburg 1982:85-86; Ridderbos 1997:512). Others hold another form of this first view, which says the Father is greater than the Son in reference to his incarnate state (for example, Cyril of Alexandria, Ambrose, Augustine; cf. Westcott 1908:2:195). Such focus on the incarnation as such or on Jesus' historical mission are quite compatible with "the belief in the unity of the divine Nature, and therefore with the belief in the equality of the Godhead of the Son with the Godhead of the Father" (Westcott 1908:2:191). Indeed, many of the fathers of the church accepted more than one view. But some also said that while the incarnate Son may be in view here, by itself this interpretation is inadequate. After all, it is no big deal to say that God is greater than a man (Basil Letter 8.5; Gregory of Nazianzus Oration 30.7).
While the words "Father" and "Son" are obviously taken from our human context, they refer, according to the second main interpretation of this verse, to realities within the Godhead itself. Fatherhood is not our projection onto God; rather it is from him that our fatherhood derives (cf. Eph 3:14-15). His fatherhood transcends our limited ideas and experience, but it is not less than that which is reflected amongst us, and indeed it provides a standard of true fatherhood. Now, to be a father one must have an offspring. Jesus is eternally Son; he is not just Son at his incarnation. Such was the faith of the ancient church, as expressed in the Nicene Creed, which refers to Jesus Christ as "the only Son of God, eternally begotten of the Father, God from God, Light from Light, very God from very God." So the Father is understood as the source of Jesus not just in his incarnation and mission, but in his eternal being as Son. "What else does the word Father signify unless the being, cause and origin of that which is begotten of him?" (Basil Against Eunomius 1.25; 3.1). The Father is greater in that he is the origin (eternally) of the Son, but he and the Son are equal in that they share the same nature (Gregory of Nazianzus Oration 30.7). To say that the Father is greater than the Son does not in the least mean that the Son does not share in the deity, since "comparisons are made between things of the same species" (Basil Letter 8.5). As D. A. Carson says, if he were to say, "`Her Majesty Queen Elizabeth the Second is greater than I,' no one would take this to mean that she is more of a human being than I" (1991:507). Thus, this passage gives a further glimpse into the relations within the Godhead without denying the oneness of the Father and the Son.
Given the focus in this Gospel on the relation between the Father and the Son it seems likely that the passage addresses this deeper issue. This does not mean that John himself was thinking in the categories the later church used to express the relation between the Father and the Son. But the fundamental mystery, the reality itself, is here revealed. The fact that the deeper relation is in view does not mean the reference to the incarnation is not also appropriate. C. K. Barrett stresses the incarnation view, but he actually captures nicely the two thoughts together in one sentence: "The Father is fons divinitatis [fountain/source of divine nature/Godhead] in which the being of the Son has its source; the Father is God sending and commanding, the Son is God sent and obedient" (1978:468).
The issues raised by this verse are matters of significant debate today. The false teaching of Arius is still quite prevalent, and thus the issue of Jesus' deity continues to be debated. But even among those who accept his oneness with God there is dispute over the nature of this relationship. Since the life of the church derives from and is to reflect the pattern of the life of God the question of hierarchy and equality within the Godhead has significant implications for our view both of God and of the life to which he calls us. Unfortunately, most of the debate seems to be between those promoting hierarchy on the one side and equality on the other. Few are wrestling with what seems to be the biblical picture of both hierarchy and equality. Fallen human society can understand hierarchy and equality separately, but to have them both at the same time is a concept found rarely if ever in fallen humanity. But then Jesus is quite clear that his kingdom is not of this world (18:36; cf. 8:23; 14:30). The patterns of kingdom life proposed by both hierarchicalists and egalitarians are altogether too much of this world. We need to take more seriously the otherworldly revelation John is passing on to us. We need now as much as ever the Paraclete to instruct us.
Jesus concludes this short section on peace by saying the very fact that he is telling them all of this ahead of time is itself a part of his message of assurance and peace (v. 29). Jesus knows what is about to occur, so therefore these events, as devastating as they will seem, should strengthen their faith in him rather than undermine it (cf. 13:19; 16:4).
After emphasizing his present teaching (vv. 25, 29), Jesus concludes by saying the time for talk is over—now come the final deeds (vv. 30-31). The reason he will not speak with them much longer is that the prince of this world is coming (v. 30). This passage has dealt mainly with the distinction between the disciples and the world, and now at its conclusion we have the fundamental contrast between Jesus and the world. Behind Jesus' human opponents is the one primary opponent who has led the rebellion that transformed the world as the created order, which was good, to the world in opposition to the loving Father. According to the NIV Jesus says this prince . . . has no hold on me. This verse may reflect a Hebrew expression (`ayin lo `ali) that was used in a legal sense of having no claim over a person (Beasley-Murray 1987:263). So Jesus would be making again the point that no one takes his life from him; rather he lays it down of his own accord (10:18). The expression has no hold on me could also be translated "has nothing in me" (en emoi ouk echei ouden). With this reading, the text would give us the reason the prince has no hold or claim on Jesus—there is nothing of his rebellion in Jesus; Jesus is not of this world (8:23).
The NIV takes Jesus' next expression as an imperative: but the world must learn that I love the Father (v. 31). The construction here (a hina clause) more often conveys purpose, and this reading would be more in keeping with the flow of thought. There is nothing in Jesus that gives the ruler of this world a hold or claim on him, but Jesus is going to go through with the Passion in order that the world may know that he loves the Father. This love for the Father is then explained in the next clause (taking the kai as epexegetic; cf. Brown 1970:656): ". . . that I love the Father, that is, that I do just what the Father commanded me to do." The command, of course, is to lay down his life, which itself is love (1 Jn 3:16).
This obedient love Jesus has for the Father is the ultimate contrast between himself and the devil. As the disciples share in this love by their own obedient love for Jesus they also will no longer be of this world (17:14). In the Passion that is about to take place, the prince of this world will be driven out, and the world will be judged (12:31). But more is involved than just condemnation. Another side of the Passion, as verse 31 reveals, is its witness to Jesus. The cross itself will demonstrate what everything else in his life has also testified, that he loves the Father and is obedient to him. Here is a manifestation for the world, and it is meant for the salvation of the world (12:32). The cross is both God's judgment and his evangelism, and both are expressions of his love. Witnessing to this revelation of the cross will be the job of the disciples, enabled by the Paraclete, as the next two chapters will explain. While Jesus will not manifest himself to the world (v. 22), the disciples' union with the Father, the Son and the Paraclete and their sharing in the divine life and peace and joy will be a witness to the world. They will bear witness both verbally and in their life to the love of God manifest in the cross.
Jesus concludes, Come now; let us leave (v. 31). These words are puzzling, because Jesus and his disciples do not seem to leave until later: "Having said these things, Jesus departed with his disciples" (18:1; obscured in the NIV, which paraphrases eipon, "having said," as "when he had finished praying"). Some commentators take the end of chapter 14 quite literally and assume the next three chapters were spoken en route, with 18:1 referring to the departure from Jerusalem (Westcott 1908:2:187). Others suggest, as commonly happens, that they stood to leave but lingered to talk further. In this case the end of verse 31 would signal a new stage in the teaching (Morris 1971:661). Others, such as C. H. Dodd, spiritualize the leaving referred to in verse 31: "There is no physical movement from the place. The movement is a movement of the spirit, an interior act of will, but it is a real departure nevertheless" (Dodd 1953:409). The majority of recent commentators believe this is a clear seam in the fabric of the Gospel, which indicates that chapters 15—17 were added to an earlier version of the Gospel. They point to the fact that 13:31—14:31 forms a coherent whole, whereas the material in chapters 15—17 shares the same style and theology and for the most part covers the same ground. Some would say, therefore, that these chapters formed an alternative version of the farewell discourse. But the fact that new angles are explored in this material (for example, through the theme of abiding and through an increased emphasis on the conflict with the world) suggests rather that this material was a supplement to the material in 13:31—14:31 and not an alternative version. This material could have been composed by later disciples, but one would expect them to have done a better job of editing. More likely it came from John, comprising further material that he was used to including as he recounted the story of Jesus but that he had left out of his draft of the Gospel. These chapters were added later either by him or by his disciples. If they were added by John's disciples, then the fact that they did not modify these last words of verse 31 to make the transition smoother may point not to their incompetence but to their reverence for their master's teaching. Thus, while there is debate about the exact nature of this inclusion (cf. Beasley-Murray 1987:223-24; Carson 1991:476-80; Paschal 1992:231-32), some such theory seems likely. Further work on ancient literary and oral forms will probably add new insight to this puzzle.
Starting your free trial of Bible Gateway Plus is easy. You're already logged in with your Bible Gateway account. The next step is to enter your payment information. Your credit card won't be charged until the trial period is over. You can cancel anytime during the trial period.
Click the button below to continue.
You've already claimed your free trial of Bible Gateway Plus. To subscribe at our regular subscription rate of $3.99/month, click the button below.
It looks like you're already subscribed to Bible Gateway Plus! To manage your subscription, visit your Bible Gateway account settings.
Upgrade to the best Bible Gateway experience! With Bible Gateway Plus, you gain instant access to a digital Bible study library, including complete notes from the NIV Cultural Backgrounds Study Bible and the Believer's Bible Commentary. Try it free for 30 days!
Three easy steps to start your free trial subscription to Bible Gateway Plus.