text:
|About Us | Press Room | Demos & Downloads | Contact Us | Site Map
Feedback in Computer-Assisted Learning
Roy B. Clariana
Clariana, R.B. (2000). Feedback in
computer-assisted learning.
NETg University of
Limerick Lecture Series.
What is feedback? In our physical environment, feedback is the return of information that is the result of an activity or process. This is often a cause-effect association, and we come to understand processes and activities both by passive observation and also by active manipulation. In our social environment, feedback is an evaluative response to our actions. Our neural systems have been tuned to allow us to note associations, both physical and social, to our survival benefit. In addition, our neural systems process language, which a fairly recent, artificial, and symbolic form.
Feedback is a natural part of human life. Our physical and social environments are full of feedback, and we are wired to use this feedback, often automatically. Also, research clearly shows that feedback is instructionally powerful. Thus, authentic instruction must bristle with overt and covert feedback information. However, after more than eight decades and hundreds of studies, there is not a generally accepted model of how feedback works in instruction (Kulhavy & Stock, 1989). There are a number of unanswered questions and perhaps even more unquestioned answers.
This chapter begins with a brief overview of the comparative effects of different forms of instructional feedback, especially those that are available in computer-assisted learning (CAL). Next, several important factors that relate to feedback effects are presented. Finally, a tentative computational model of feedback is presented based on associative learning and connectionist approaches. A visual metaphor termed high-dimensional semantic space is provided to describe the effects of feedback.
In CAL, learner response is generally limited to recognition (clicking on) and recall (typing in). Fairly soon, CAL will be capable of properly responding to extended learner input, such as a 200-word essay response utilizing automated approaches such as Latent Semantic Analysis (Foltz, Kintsch, & Landauer, 1998). However, currently, CAL feedback research must focus on recall or recognition learning outcomes; and CAL feedback is fundamentally limited to something that can happen on the screen (and sometimes audio). This would include displays of words, numbers, highlights, motion, animations, arrows, icons, color changes, and other cues.
Traditionally investigated types of feedback available in CAL include the following: Knowledge of response (KR) that states "right" or "wrong" or otherwise tells learners whether their response is correct or incorrect; Knowledge of correct response (KCR) that states or indicates the correct response; and Elaborative feedback that includes several more complex forms of feedback that explains, directs, or monitors (Smith, 1988). Elaborative feedback includes the forms listed below:
All of these are specifically and intentionally directed at individual learner responses except for monitoring feedback, which provides information over a group of learner responses, such as over a section, a lesson, or an entire unit of CAL instruction.
Advisement is worth a brief diversion. As mentioned, monitoring feedback or advisement (Tennyson, 1980) is meta-level feedback, such as ongoing on-screen lesson scores, recommendations for instructional branching, and end-of-unit and end-of-lesson scores. For this reason, advisement may primarily affect continuing motivation rather than item-specific error correction.
How does advisement function? Current advisement research views advisement forms within a continuum of effectiveness that relates to the information quality and quantity provided by the advisement. Advisement relates to instructional decision-making by the learner, and much CAL advisement research includes learner- versus program-control conditions (Coorough, 1991; Hannafin, 1984; Johansen & Tennyson, 1983; Santiago & Okey,1992; Tennyson & Buttrey, 1980). Learners usually appreciate this sort of information although they don't always use it appropriately.
Advisement effectiveness results are mixed. Milheim and Azbell (1988) reported that advisement, in general, improves posttest scores, increases the number of students that reach mastery, increases time on task, and increases instructional efficiency. Clariana (1992, 1993a) reported that advisement in CAL positively effects persistence, lesson production, attendance, and achievement. Thus, advisement should be provided to learners in CAL.
Feedback Timing
Different forms of feedback like those mentioned above can be provided immediately after a learner's response; or feedback information can be delayed for some amount of time such as seconds, minutes, or days after the learner's response. Feedback timing is important for at least two reasons. First, because our neural systems are temporally based, immediate feedback is likely to be processed differently than delayed feedback (Shanks, Pearson, & Dickerson, 1989). Second, previous research has shown performance differences for immediate and delayed feedback.
Two different models may explain the differences between immediate feedback (IF) and delayed feedback (DF), the Interference Perseveration Hypothesis (IPH; Kulhavy & Anderson, 1972) and a dual-trace information processing explanation (Glover, 1989; Kulik & Kulik, 1988). The IPH depends on initial lesson errors interfering or not interfering with learning the correct response. IPH holds that with IF, the error memory trace activation interferes with encoding the correct response memory trace. The delay with DF allows the initial question and error response association to subside before the association between the question and the correct response is formed or strengthened. In addition, with the IPH, interference occurs very late during encoding and likely requires overnight consolidation.
Alternately, the dual-trace explanation holds that two processing opportunities occur with DF and only one with IF. Thus, DF groups will remember both their initial error response as well as the question–correct response associations on the retention test. For the IF group, the question–error response association is weakened and the question–correct response association is strengthened. Therefore, with the dual-trace explanation, differences between DF and IF are due to retrieval response competition between the dual memory traces under DF compared to no retrieval competition for the single memory trace under IF. The trace with the greatest activation level would be selected as the correct answer. In a study of IF and DF, Clariana, Wagner, and Roher-Murphy (2000) reported that DF groups remembered their initial errors significantly better than the IF group 24-hours later. This supports the dual-trace explanation. Also, no difference was observed between the IF and DF groups in 24-retention of correct response scores. Remembering initial lesson errors did not relatively improve the DF groups' retention test score; in fact, errors perseverated under DF.
Many feedback timing studies and meta-analyses of studies provide guidelines for when to use DF or IF (Bangert-Drowns, Kulik, Kulik, & Morgan, 1991; Kulik & Kulik, 1988). For example, in situations that use questions only with feedback as instruction, such as in a CAL drill, DF is superior to IF with an effect size of 0.36. However, in studies that use questions with feedback and include additional instructional materials, such as in a CAL tutorial, IF is superior to DF with an effect size of 0.28 (Kulik & Kulik, 1988). These are only general guidelines. There is usually little difference in learning for immediate versus delayed feedback, though Clariana et al. (2000) recommended that DF be used when it is advantageous to remember both initial error responses and correct responses, such as in CAL simulations. For example, Lewis and Anderson (1985) reported that learners receiving DF in an adventure game were better at detecting errors relative to the IF group. However, when it is critical to inhibit, suppress, or forget errors, such as in CAL typing skills programs and when acquiring speeded or automatic responses involving critical, difficult, or dangerous content, IF should be used.
Comparative Feedback Effectiveness
Which type of item-specific feedback is best? Some studies have shown KCR superior to KR, while KR was superior to no feedback (Gilman, 1969; Kulhavy, 1977; Travers, Van Wagenen, Haygood, & McCormick, 1964; Waldrop, Justin, & Adams, 1986); however, the results have been mixed (Schimmel, 1986). A meta-analysis by Bangert-Drowns, Kulik, Kulik, and Morgan (1991) compared KR, KCR, AUC, and elaborative feedback to no feedback. Compared to no feedback, KR feedback was least effective (e.s. = -0.08), followed next by KCR (e.s. = 0.22). AUC, (e.s. = 0.53) and elaborative feedback (e.s. = 0.53) were equally most effective. The performance results may be summarized as:
KR < no feedback < KCR < AUC = Elaborative feedback
In general, more feedback information means more learning.
A review of 30 studies by Clariana (1993b) that compared no feedback, KR, KCR, and DF to multiple-try feedback (MTF, mostly AUC) reported a similar superiority for MTF over no feedback, e.s. = 0.56. However, unlike Bangert-Drowns et al., he reported no difference between KR, KCR, DF, and MTF. These results may be summarized as:
no feedback < KR = KCR = DF = MTF(AUC)
This finding is consistent with a meta-analysis by Schimmel (1983) involving 15 studies that compared KR and KCR to no feedback. KR and KCR were each more effective than no feedback (e.s. = 0.47). Clariana's and Schimmel's findings can only be summarized as any feedback is better than no feedback, a rather disappointing conclusion after having come so far.
Factors Related to Feedback Effectiveness
Then, besides DF, are all forms of feedback about equivalent? At least differences in DF and IF can be readily explained (see above) with regard to the number of memory traces established or altered by feedback. Several other factors are emerging that may explain when certain kinds of feedback are more effective. These include learner prior knowledge, the intended kind and level of learning outcome, and the separate contribution of question effects in relation to overall feedback effects. Note that all of these directly relate to lesson item difficulty. These factors will later provide a bridge to describe a model of how feedback works.
A Possible Trait-Treatment Interaction with Multiple-Try Feedback
Clariana (1993b) reported a trait-treatment interaction (TTI) relating MTF and prior knowledge. Specifically, for low-prior-knowledge students, KCR was more effective than MTF, e.s. = 0.11 (see Figure 1). For high-prior-knowledge learners, MTF was better, e.s. = 0.39 (Clariana, 1999a).
Figure 1. Interaction of prior-knowledge and type of feedback (Clariana, 1999a).
Since MTF and KCR feedback information is usually identical when the learner's response is correct, this TTI likely relates to affective and cognitive differences on errors. An affective explanation of reduced effects for MTF is straightforward. If a learner makes many lesson errors, then frustration is likely and MTF would have a negative effect. This negative effect for MTF would increase as the number of errors increases. Thus, MTF would likely obtain a relatively negative effect with difficult or new lesson content and with low-prior knowledge students.
For example, Dick and Latta (1970) reported that low-ability students became frustrated during a repeat-until-correct lesson using constructed response study tasks relative to high-ability students. Morrison, Ross, Gopalakrishnan, and Casey (1995) reported that AUC students' selection of review screens deteriorated in relation to the number of lesson errors committed. Further, they reported that the AUC treatment groups' lesson scores were considerably lower than the lesson scores for KCR and other groups. Similarly, Clariana (1999a) reported increased frustration by students in an MTF treatment relative to students in a KCR treatment. During the lesson, students in the MTF treatments pressed the Enter key without trying to respond at nearly three times the rate of the KCR group, thus obtaining lower lesson scores than the KCR group.
Alternately, some learners such as high-ability and/or high-prior knowledge students may prefer MTF, which should positively impact learning. For example, Clariana and Lee (2001) reported that 71% of learners (graduate education majors) preferred MTF to KCR with multiple-choice lesson tasks. Regarding MTF, the students wrote comments like "I learn more when I get it on my own" and "Trying a second time makes me think more about the question and what it is really asking." This positive effect for MTF should be more likely for easier, familiar, personal, and interesting lesson content and for high-prior-knowledge students (Nishikawa, 1985), and with recognition study tasks relative to recall study tasks. Thus, a TTI for MTF can be readily accounted for by increased or decreased engagement in the lesson due to motivational issues related to learner characteristics and the lesson content.
From a cognitive amount-of-information viewpoint, MTF essentially provides the same information as KCR. However, MTF requires information from within the learner while KCR information is presented from outside of the learner and as true. When an error occurs, MTF information acts to alter the existing mental representation of the item and response, perhaps directly, allowing the item and correct response to fit within the learner's broader preexisting understanding of this content. Alternately, KCR information acts to reject the preexisting mental structure and provides the item and correct response as a new instance of this content. Thus, MTF and KCR, though outwardly similar, have very different effects on episodic memory.
A number of studies have shown that MTF is only equal to or is relatively less effective than single-try forms of feedback, such as KR, KCR, and DF, when posttest questions are identical (as repeated memory instances) to lesson questions (Anderson, Kulhavy, & Andre, 1971; Bardwell, 1981; Clariana & Smith, 1989; Clariana, Wagner, & Roher-Murphy, 2000; Hodes, 1984; More, 1969; Morrison et al. 1995). However, when posttest questions are different than the lesson questions (not as repeated instances, but meaningful), such as trouble-shooting new problems, applying principles, and classifying new concept examples, then MTF is often relatively more effective than single-try forms of feedback (Angell, 1949; Cantor & Brown, 1956; Clariana, Ross, & Morrison, 1991; Dempsey & Driscoll, 1989; Lee, Smith, & Savenye, 1991; Nielson, 1990; Pressey, 1950, Richards,1989).
For example, Clariana et al. (1991) examined the effects of several forms of feedback on identical and higher-level learning outcomes. High school students (N=103) in a summer remedial program completed a difficult computer-based lesson on science principles with text portions, verbatim and inference adjunct lesson questions, and one of three forms of feedback including DF, KCR, or MTF. Posttest questions were identical, paraphrased, and transformed following Anderson (1972), Bormuth (1970), and Bormuth, Manning, Carr, and Pearson (1970) in order to measure higher-order learning outcomes. All three feedback groups performed significantly better than the no-feedback control on identical retention test items (repeated instances, rote learning), but there was no difference between the feedback and control groups for higher-order posttest questions. However, note that the MTF scores were relatively lower than DF and KCR scores for identical posttest items, but MTF scores were significantly better than DF scores for paraphrased and transformed-paraphrased questions. This lends support to the interpretation that MTF is better than single-try forms of feedback (at least DF) with higher-order learning outcomes (see Figure 2).
Figure 2. MTF (triangles, dashed line), DF (diamonds), and KCR (boxes) immediate posttest scores from Clariana et al. (1991).
This finding is hardly surprising. According to Bransford (1979) and Jonassen (1985), instructional activities that require the learner to actively generate a response from their own internal understanding should be more powerful than activities that only present information, where meaning comes from outside of or external to the learner. Clariana et al. (1991) states,
Allowing unassisted multiple response tries has considerable intuitive appeal. Answering until correct may engage learners in additional active processing following errors and also ensures that the last response is a correct one, a principle espoused over half a century ago in the contiguity theory of Edwin Guthrie. (p. 6)
Therefore, high prior-knowledge learners, by definition, would have meaningful or more developed mental representations or schemas relative to low prior-knowledge learners; and MTF can take advantage of a rich schema while KCR cannot. Thus, MTF tends to act on an individual's internal and personal meaning of the lesson content, and DF, KCR, and other forms of feedback that present the correct response to the learner tend to establish strong specific instances of the content. If so, MTF should logically have a greater effect on higher-order, meaning-related learning outcomes, and DF and KCR would lock down facts as memory instances. This explanation can account for both hypotheses, the TTI for MTF and prior-knowledge, and MTF effects on higher-order learning outcomes.
Next, we provide a description of the effects of adjunct questions without feedback and of text. This will allow us to later separate the effects of text, questions, and feedback.
Adjunct Questions Without Feedback
A lot is known about adding questions without feedback to instructional text (Hamaker, 1986; Lindner & Rickards, 1985). For example, either providing questions before text, inserted within text, or massed after the text is effective. However, questions inserted in the text are usually most effective and probably influence the processing of both concurrently read text material as well as to-be-read text material (Andre, 1979; Duchastel, 1983; Hamaker, 1986). When comparing the response formats, short answer constructed response lesson questions have a stronger effect than multiple-choice lesson questions on both constructed response and multiple-choice posttests questions (Anderson & Biddle, 1975; Hamaker, 1986). Additionally, it is hardly surprising that posttest performance is higher when study time is adequate and the text is continually available. These are all ecologically valid suggestions that should be considered during the design of CAL.
Perhaps most significantly, higher-order lesson questions usually obtain larger effects than factual-level lesson questions for both lower-level learning outcomes, such as facts and verbal information, and also for certain higher-level learning outcomes, such as applying knowledge of concepts and principles in order to recognize new examples, and to solve problems involving the same concepts or principles. However, the relatively positive effect of higher-order lesson questions usually does not generalize to new/other concepts or principles (Andre, 1979; Hamaker, 1986; Lindner & Rickards, 1985). Also, as with MTF, a TTI has been suggested for adjunct questions and prior knowledge (Hudgins, Dorman, & Harris, 1979; Lindner & Rickards, 1985). Poor readers do better with frequently inserted questions, which may serve to place instances, mileposts, or anchors to populate and structure their mental representation with the new content. In contrast, good readers do better with post text questions, which may serve to refine or tune their already existing mental representations of the text. In summary, lesson questions without feedback, and especially higher-order lesson questions, are instructionally effective and should be used in CAL.
The Contribution of Questions and of Feedback to Higher-Order Learning
Unfortunately, the feedback literature is not as extensive or clear as the adjunct question literature on the effects of different forms of feedback on higher-order learning outcomes (Mory, 1996; Smith & Ragan, 1993). Much of the feedback literature has dealt with lower-level verbal learning outcomes (Merrill, 1985; Mory, 1996; Schimmel, 1983), and the approaches often involved multiple-choice lesson questions that were repeated in some form on the posttest. However, even though positive feedback effects are well established for lower-level learning outcomes, it is unclear that feedback can effect higher-order learning outcomes (Mory, 1996).
Morrison et al. (1995) compared several forms of feedback (DF, KCR, and MTF) and also included text-only and questions-only control groups. This allows us to determine the separate effects of questions and questions plus feedback for lower and higher-order learning outcomes. Undergraduate education majors (N=248) completed a computer-based lesson on writing instructional objectives. Incentives were manipulated by counting posttest achievement as a part of the student's course grade in one condition, but in the other condition, participation without regard to posttest achievement was part of their course grade. It was expected and also observed that students in the grades condition would try harder to learn the lesson material relative to those in the no-grades condition. The posttest question-level design followed Clariana et al. (1991) using identical, transformed, and paraphrased questions. The results for the feedback and the question treatments are shown in Figure 3 in terms of effect sizes relative to the text-only control group.
For identical-level posttest questions (rote), both questions (e.s. = .52) and questions with feedback (e.s. = .75) had a strong effect. The question portion accounted for about 70% of the question plus feedback identical-level posttest scores. For transposed and paraphrased posttest questions, questions accounted for about 77% of the question plus feedback posttest scores. For combined transposed plus paraphrased posttest questions, the feedback actually subtracted from the effects of the question portion.
In a study by Clariana (2001) using higher-order lesson questions, questions accounted for an even greater proportion of the questions plus feedback effects (see Figure 4). High school students (N=78) were randomly assigned to one of five lesson treatments: text plus questions with DF, text plus questions with KCR, text plus questions with MTF, text plus questions with no feedback, and a text-only control group. A retention test given five days after instruction assessed both verbatim memory of principles from the lesson and also higher-level understanding of principles from the lesson (again paraphrased and transformed).
Figure 4. Effect size comparisons from Clariana (2001) showing feedback treatments (solid lines) and questions treatment (dashed lines) relative to the text-only control (where effect size equals 0).
Relative to the control group, feedback was beneficial for lower-level learning but was detrimental for transformed test items (see Figure 4). For identical-level posttest questions (rote), both questions (e.s. = .48) and questions with feedback (e.s. = .56) had a strong effect. The question portion accounted for about 85% of the question plus feedback identical-level posttest scores. For paraphrased posttest questions, questions accounted for about 50% of the question plus feedback posttest scores. For transposed and the combined transposed plus paraphrased posttest questions, the feedback again substantially subtracted from the effects of the question portion. Transposing posttest questions had a strong negative effect on all treatments.
The major difference between the Morrison et al. (1995) results and the Clariana (2001) results occurred with transposed (T & TP) posttest items (compare Figures 3 and 4). Why are the results for transposed posttest questions so different? The Morrison study mainly involved concept identification and definition while the Clariana study involved science principles, described as the interrelation between several concepts. Transposed stem and response for these two learning outcomes obtain very different forms of posttest questions. For example, from Morrison, "An objective is a statement of what the learner will be able to do…" transposed becomes "A statement of what the learner will be able to do is an objective." In this case, transposing concept identification and definition type questions produced nearly equivalent questions.
However, transposing principles does not produce equivalent questions, and in some cases, transposing a principle will produce a non-truth. For example, from Clariana, "increasing heat in a closed cylinder increases pressure" transposed becomes "increasing the pressure in a closed cylinder increases the heat." Though both are true, the first is truer than the second because heat will escape the closed system though pressure cannot. Similarly, Kintsch (2000) has shown that another form of principle, namely a metaphor such as "My lawyer is a shark," also is not reversible (cannot be transposed). Therefore, principles often should not be transposed and logically, our neural systems should not automatically allow these broader associations.
In addition, our internal neural networks act to reduce this type of over-generalization by balancing inferences produced by a principle with real-world experiences of related instances. Instances, especially if part of personal experience, can win out or dominate processing relative to generalization from learned principles. Concepts are closer to the experienced world than are principles, which are composed of concepts. Instances and examples that are experienced are likely to dominate the understanding of a principle. This is often referred to in the science education literature as naïve misconceptions, such as the sun rises rather than the earth rotates under the sun. In the heat-and-pressure case above, common experience suggests that a soft drink can, which has increased internal pressure, usually feels cold, not warm. Thus, it is easier to believe that soft drink cans are cold than to believe that pressure increases heat in the can. The instance wins out, and the principle does not transpose.
Transposing and paraphrasing posttest questions provide particularly important measures of associative learning. These two studies, and our argument so far, strongly indicate that feedback mainly involves a highly specific question and response association. A recent study by Clariana, Wagner, and Rorher-Murphy (2000) has applied an associative learning, connectionist approach to describe feedback effects. Their approach and results will be described in more detail since it is central to our explanation of feedback effects.
A Connectionist Description of Feedback Timing Effects
Recently, Clariana (1999b) has suggested that an associative learning, connectionist approach can be used to predict the quantitative effects of feedback on posttest memory activation levels of errors and of correct responses (Haberlandt, 1997; Elman, 1993; McLeod, Plunkett, & Rolls, 1998; Plunkett & Marchman, 1993; 1996; Seidenberg & McClelland, 1989). He hypothesized a feed-forward network using the delta rule to describe feedback effects with multiple-choice lesson questions (see Figure 5). A feed-forward network consists of input nodes connected to multiple output nodes (Shanks, 1995). The input node represents the multiple-choice question. The output node would represent possible responses. The alternative with the greatest activation level represents the response the neural network will give to that input pattern.
Figure 5. A simple connectionist network with several possible output activation levels (aout) for a given input pattern (ain).
Feedback in this model acts to strengthen question–response associations that are correct and to inhibit associations that are incorrect. The amount of increase or inhibition was computed using the delta rule. The delta rule (Widrow & Hoff, 1960) describes the change in association weight, termed Dw, between the input unit and output unit at each learning trial, as:
Dwio = a ain (to - aout)
The term a is the learning rate parameter (a fudge factor constant that dampens the impact), ain is the activation level of input units, to is the desired response (the t refers to "teacher", in this case to comes from item feedback), and aout is the activation level of the output units (Shanks, 1995). In instructional terms, learning is an increase in the association, which is an increase in Dwio between the stimulus (ain) and the correct response (aout), with a relative decrease in association, which is a decrease in Dwio for incorrect responses.
To apply the delta rule, Clariana (1999b) assumed that lesson average item difficulty values are reasonable estimates of the association weights of the correct responses (aout). Item difficulty (p) is defined as the proportion (pg) of individuals who answer an item (g) correctly (item difficulty notation convention from Crocker & Algina, 1986). For example an item difficulty of .20 indicates that 20% of the learners responded correctly to that item. Item difficulty values can range from 0.00 to 1.00 with low values indicating difficult items and high values indicating easy items.
In the delta rule equation, feedback impacts learning in the term (to - aout). Customarily, the values for to and aout are constrained between 0 and 1 (McLeod, Plunkett, & Rolls, 1998). The value for to equals 1 if the activation level of the input unit matches the desired response (i.e., with correct responses), and to equals zero if the activation level of the input unit does not match the desired response (i.e., with incorrect responses). Therefore, with correct responses, the association weight increases since (1 - aout) is positive while with incorrect responses, the association weight decreases since (0 - aout) is negative. In other words, when feedback is provided as part of the responding instance, correct responses are strengthened and incorrect responses weakened.
Thus, given lesson item difficulties as initial aout, to equal to 0 or 1, and a ain = 0.4 (per Clariana, 1999b), the delta rule can be used to predict posttest item difficulties, aout after immediate and delayed feedback (see Figure 6).
Figure 6. Predicted retention test values generated by the delta rule across a range of lesson item difficulty values (from Clariana, 1999b). Predicted retention test memory of Initial Lesson Responses (ILR) are shown as dashed lines and Correct Responses (CR) are shown as solid lines for delayed feedback (DF) and immediate feedback (IF).
Clariana, Wagner, and Rorher-Murphy (2000) tested this computational model. High school students (n = 52) completed a computer-based lesson with text portions, verbatim and inference-level recognition study tasks, and either DF, STF (KCR), or MTF. The retention test given 24-hours after the lesson consisted of items identical to the lesson. On the retention test, students were asked to remember their initial responses from the day before and also the correct responses. There was a significant difference for type of feedback, with retention test memory of initial lesson responses greater under DF than under STF (see the left panel of Figure 7). These results closely match the predicted inhibition (or not) of initial lesson response errors for immediate feedback relative to DF. This shows that the computational model was able to predict inhibition of lesson errors fairly well; however the model over-estimated increased activation of correct response values (see the right panel of Figure 7), suggesting that different mechanisms are at work with question–correct response associations.
These findings show that both the direction and magnitude of feedback effects, at least for lesson errors, can be computationally determined. This suggests that automatic processes may be at work. Also, though lesson errors are important since they reflect a learner's mental representation of a question–response association, previous models have not systematically accounted for lesson errors. For these reasons, we advocate that feedback researchers consider an associative learning, connectionist approach.
Explaining the effects of text, questions, and feedback
Below is a visual model of mental representations. This visual model is used to describe the effects of text, questions, and feedback on learners' mental representations of CAL lesson material.
Visualizing Mental Representations
Mental representations can be visualized in several ways. Collins and Quillian (1972) proposed semantic networks that use nodes to represent concepts and lines between nodes to show the relationships between concepts. For example, some relationship data from McClelland (1981) that describes two fictitious gangs, the Sharks and Jets, can be displayed as a semantic network (see left panel of Figure 8). For descriptive purposes, added information about Gangs and about Sharks and Jets not in the original data set was added to fill out the semantic network. The Shark's gang member named Dave is a divorced drug pusher. Semantic networks have been used to describe many aspects of information, such as the hierarchical structure of the information.
Figure 8. A semantic network (left) and a MDS (right) of associations for two fictitious gangs (from McClelland, 1981).
Graphical displays of psychological space have also been called high-dimensional semantic space (HDSS; Foltz, Kintsch, & Landauer, 1998) and cognitive maps (Diekhoff & Wigginton, 1982). Several recent computational approaches show similarity in mental representations as distances in psychological space rather than as nodes and links (Diekhoff & Wigginton, 1982; McLeod, Plunkett, & Rolls, 1998). These approaches provide another way of displaying and of thinking about an individual's mental representation of information.
A scaling procedure, such as multi-dimensional scaling (MDS), can display relationship data visually in fewer dimensions. For example, to use MDS to display HDSS, first relationship data is described in a weight matrix (see Table 1). A 1 in the matrix indicates an association between the column and row instance, while a 0 indicates no association. Each row (or column) in the table is a high-dimensional vector. Next MDS can be applied to that matrix. MDS was conducted with Table 1 data using SPSS 9.0 including the standard default values for MDS except for selecting create Euclidean distances and selecting display group plots (see right panel of Figure 8).
Table 1. Weight matrix of some of McClelland's (1981) data (with 8 examples of gang members).
First, note that Art is nearer Clyde than Alan (see right panel of Figure 8) showing that the Art and Clyde (with four overlaps) have more similar weight vectors (see Tables 1 & 2) than Art and Alan (with only two overlaps). Simple correlations of vector elements from Table 2 for Art, Clyde, and Alan provide the same information: Art vs. Clyde, r = .54; Art vs. Alan, r = .08; and Clyde vs. Alan, r = .08.
Table 2. Vector elements for Alan, Art, and Clyde.
Next, Art is located near single, bookie, and Jets, as well as Junior High School, and 40-ish. Next, note that in this MDS representation (see Figure 8), Dave is nearest divorced, pusher, and Sharks. Category data from the original data set, such as age 20-ish and High School graduate, are shown as instances along with gang-member instances. Data information that all Sharks share, such as hanging on the north-side and wearing blue, stack precisely on top of Sharks in the MDS, since Sharks, blue, and north-side all have identical weight vectors. In this sparse MDS with only 8 gang members, instances of gang members and characteristics of the instances are all mixed together. Note how the MDS changes when more instances of gang members are included in the matrix (in this case 27 instances, see resulting MDS in Figure 9).
Figure 9. MDS of McClelland (1981) data including 27 gang members. Top and bottom panels are identical, except the bottom panel has been stretched to display details.
The gang member instances grouped together in a cluster or cloud because of the relative similarity of their weight vectors while the vectors that characterize the instances, such as pusher, bookie, and burglar, move away on the dimensional scales (see upper panel of Figure 9). The concept of gang member is represented in this HDSS as a cloud or cluster of instances of gang members, even though there is no weight vector called gang member in the data set to cause this grouping. The cloud of instances is highly structured. Similar gang members will be nearer each other because their vectors are similar. For example, why are Doug and Nick near each other (see center of the bottom panel of Figure 9)? At first glance, they seem dissimilar since Doug is a Jet pusher and Nick is a Shark bookie. However, since Doug and Nick are both 30-ish, high school graduates, and single, they actually have a lot in common.
HDSS as Structural Knowledge
The association between instances in HDSS looks like structural knowledge, which is the knowledge of the inter-relationship of knowledge elements. It has been suggested that developing structural knowledge of a content area is necessary in order to flexibly use that knowledge-base (Jonassen & Wang, 1992). This MDS approach visually depicts structural knowledge (Diekhoff & Wigginton, 1982) and demonstrates that hierarchical structural knowledge can emerge in HDSS when enough cases or examples (vectors) are included in the weight matrix. This visual approach would further suggest that if you directly teach structural knowledge relationships before multiple instances are established, then the structural knowledge associations would simply be another vector like gang member or pusher or Alan. Thus, teaching structural knowledge relationships too soon would result in relatively weaker and even possibly meaningless one-vector effects on HDSS rather than regional (cloud) multi-vector effects that would be quite meaningful to the learner.
If structural knowledge emerges in actual mental representations in a similar way to this MDS model, then CAL should present many cases or examples per coherent knowledge base (in this example, the cluster did not emerge until at least 22 gang members were included in the MDS matrix). Probably human neural networks are more capable of deriving structural knowledge than this MDS approach, suggesting that fewer cases would be required for the emergence of structural knowledge. Nevertheless, learners can only interact with existing instances in order to generate structural knowledge (Jonassen & Cole, 1993). Given the likely importance of structural knowledge, it is reasonable in CAL to include many cases and examples and then require the learner to interact with the cases.
Note that HDSS is a model of association weights, not of activation weights. This is an important distinction. The entire weight matrix is probably never activated at the same time, only parts are activated. An input pattern, such as a question, interacts with the entire weight matrix to produce a network of activation that is a subset of HDSS, and the HDSS response to the question would be one instance or several instances within that subset. For example, if the learner were asked to list all Jets who are burglars, first Jets and burglar would activate (see Figure 9). Note that Jets and burglar, besides being instances in HDSS, are also an element of every other vector in the matrix. Any other instance in the matrix that has a 1 in the Jets and/or burglar element of its vector also activates to some degree. This process has been referred to as spreading activation and will include nearest neighbors in HDSS, such as George, Lance, John, Jim, and possibly Alan (also a correct response) and Ken (an incorrect response). This is a likely subset of HDSS that is activated by the question.
Next, this subset is reduced to one or just a few instances. Kintsch (1994) has described a process of how the activation level of only one or a few instances in a weight matrix increases as the question vector cycles upon the weight matrix while activations of all of the other instances decrease (relatively). A small number of instances with the greatest activation levels when the pattern stabilizes, such as George, John, and Lance, are the HDSS response to the question.
In addition, a vector with a one-element association is established between the question pattern "list all Jets who are burglars" and the response pattern "George, John, and Lance." If the same question is asked later, the response will be considerably quicker because it involves vectors with fewer elements, which will cycle faster. This process could be called rote responding. It is also referred to as the familiarity-based process of recognition, which stabilizes in about 200ms (Rotello & Heit, 1999), compared to the slower (800ms to 2000ms) recall-like or recollection process that involves many more vectors representing more qualitative information and requiring more cycles for the pattern to stabilize (Dobbins, Kroll, Yonelinas, & Liu, 2000). If the system doesn't immediate identify an input pattern, then it resorts to the slower recollection process. It is not surprising that our neural system would act this way, since there is obviously a powerful survival benefit in shortening the response time to danger.
This vector-based approach is easy to apply and understand and is highly explanatory. However, it oversimplifies how a question is handled by an individual's mental representation. First, it reduces weight matrix associations between instances to either a 1, 0, or –1; and there is possibly a gradation in associations. Second, the model assumes a relatively naïve HDSS since it tends to disregard preexisting association data. Regarding gradation in association, many neural-systems rely on sigmoid activation functions which drive activation towards 1, 0, or –1 (McLeod, Plunkett, & Rolls, 1998); therefore, this simplification may not be a problem. Regarding the naiveté of the HDSS representation, a relatively uncluttered area of HDSS can be established for new content by simply adding vectors of unique instances, such as context characteristics. For example, the more unique context vectors that are included, the further the entire content representation is driven away from preexisting content instances and thus into clear HDSS space (new dimensional space). This easily accounts for context differences in recall memory, such as a list learned underwater is not as well remembered on land and vice versa (Godden & Baddeley, 1975, 1980). The underwater context variables, which are many, compartmentalize the list words together within a generally less accessible area of HDSS when recalling the list on land. Reestablishing context variables, such as wearing a mask and breathing through a mouthpiece while on land, should tend to bring up the underwater list.
In addition, instances and associations (the elements in an instance) in HDSS may have idiosyncratic activations. For example, if Ken was just previously mentioned, the residual activation from that thought would add to the weak activation that results from the "Jets who are burglars" question. Thus, Ken would become active enough to emerge as a correct answer, though Ken is not a Jet. Also, specific instances in HDSS may have higher baseline activation than other instances, probably due to individual familiarity with those instances. Idiosyncratic activations would account for idiosyncratic responses to text, questions, and feedback.
Using HDSS to Describe the Effects of Text, Questions, and Feedback
The effects of altering a vector element on HDSS can be used to explain the effects of text, questions, and feedback. First, reading a text loads HDSS with additional content by changing 0s to 1s in the weight matrix for associated information, such as "Doug is a high-school graduate." Some instances and associations may already be filled in due to prior-knowledge and experiences. Note that unless associations are specifically stated or implied by the text, there are biologically conservative reasons why the remaining elements in the vector will be 0. In this case, the effect of replacing a 0 with 1 is to make the Doug vector and the high school graduate vector more alike, therefore closer together in HDSS.
Like word instances, propositions and sentences will also be placed as single instances in HDSS (Kintsch, 1994). Foltz, Kintsch, and Landauer (1998) reported that text coherence can be improved by making sure that consecutive sentences have semantic overlap, either by repeating words or repeating synonyms of words in successive sentences. This type of semantic overlap should allow for consecutive sentences to obtain more similar vectors and thus be nearer each other in HDSS, which intuitively seems like a good idea. Low prior-knowledge readers benefit the most from highly coherent text (Britton & Gulgoz, 1991; McNamara, Kintsch, Butler, & Kintsch, 1996). Unfortunately, this type of writing style can be wordy and boring, especially for high-prior knowledge readers.
Questions add to the effects of text in at least two ways. Questions can just repeat text information or can cause the reader to make inferences from the text. When a question repeats text information, it should simply strengthen the association between the instances, changing a 0 vector element to 1, similar to the text case above. In the same way, text summaries should have the same effect as questions that repeat the text.
Question can also drive inferences. For example, a text may say in one place that Doug and Nick had a big argument after their graduation and, later, that they were 18 at the time. The inference is that they graduated from high school. A question could ask, "Did Nick graduate from high school?" This question would act to strengthen and complete the weight matrix by associating several separate instance vectors, thus establishing a more accurate representation in HDSS.
Does requiring the learner to make the inference on their on result in a stronger memory trace association? In other words, do questions result in more fully associated and thus better and more accurate HDSS? As described earlier in the chapter, previous research has shown that questions, and especially higher-order questions, have a stronger effect than lower-order questions and than text only. Questions are fundamentally different than text in that questions probe and activate the learner's existing mental representation. Therefore, questions tune the HDSS as long as the learner is able to answer the question correctly.
How does feedback complement the effects of questions? Immediate feedback has been shown to strongly inhibit error associations (Clariana et al., 2000). What does this mean in terms of HDSS? Suppose that the learner was asked whether Doug was a Shark or Jet. Note that Doug is near the boundary for Shark (see center of the bottom panel of Figure 9). If Shark were primed at that point due to preceding activity, the learner could reasonably though incorrectly respond that Doug is a Shark. Without feedback, that incorrect association between Doug and Shark would be strengthened. The Doug vector would point more towards Shark territory, and the learner would more likely repeat this error association in the future.
However, if immediate verification feedback (stating yes, no, right, wrong, and so on) is provided, it would inhibit the direct association between Doug and Shark. Since that association is 0 in the weight matrix table, then it would become negative. When the MDS is recomputed with a -1 association for Doug and Shark, Doug moves away from Shark territory and deeper into Jet territory just beside Clyde. This new MDS suggests that immediate verification feedback can establish an orthogonal relationship between Doug and Shark by altering only one element of the Doug vector.
The effect of immediate verification feedback on HDSS can be weak or strong. If there are many examples related to Doug in HDSS, such as should occur with high-prior knowledge learners who have read and understood the text, then these instances will tend to hold Doug's position as is; and feedback has a relatively smaller effect. But if the HDSS is sparse, such as with low-prior knowledge students who did not read or else did not capture the text, then there are fewer instances holding Doug in place; and Doug will be moved quite a distance in HDSS. Therefore, immediate verification feedback only slightly alters or tunes associations of a rich mental representation but can substantially alter a sparse mental representation.
Multiple-try feedback usually provides verification and with errors, states "try again." Like repeated questioning, providing MTF to high-prior-knowledge learners further enriches an already rich psychological landscape by slightly adjusting the positions of existing content, especially by establishing orthogonal relationships for error associations. Knowledge of correct response feedback, after providing verification, then provides the correct response. KCR establishes a familiarity-based fast association between the question pattern and the response pattern. For posttest questions that are identical to the lesson questions, this is good. However, for more meaningful and new posttest questions, familiarity-based associations bias an already rich landscape with strong distractions. The associations compete with the meaningful existing structure.
On the other hand, low-prior-knowledge learners have relatively flat or unpopulated mental representations for that content. Therefore, additional processing tries required by MTF would involve an HDSS that cannot activate a correct or meaningful response. This would be both useless and frustrating since MTF would tend to scramble up the few correct associations that exist as the learner tries and rejects existing instances, disassociating each instance in turn from other existing instances. No wonder MTF obtains negative effects with low-prior knowledge learners. Therefore, immediate KCR feedback is best for low-prior-knowledge students. KCR provides question–correct response associations to populate schema with familiar instances, and those instances would tend to dominate mental representations of the content.
In CAL, we must do a better job of designing instructional text. Text coherence related to semantic overlap between sentences has been shown to be critical for learning from text. Since the limited screen area in CAL relative to print reduces re-reading of previous sentences (Clariana & Smith, 1988), text coherence must be greater in CAL than with paper-based text.
Questions are effective and account for a substantial portion of the effects of questions with feedback. Questions apparently establish more accurate mental representations of text content. Thus, providing questions in CAL, and especially higher-order questions, is a good idea as long as the learner is able to correctly answer the question. When the learner answers a question incorrectly, then feedback should be provided.
Regarding feedback, here are several tentative conclusions:
If the conclusions of this paper are confirmed, then the effects of feedback are fairly direct and can be predicted. However, substantial research remains to be done to validate and confirm the views expressed above.
Anderson, R. C. (1972). How to construct achievement tests to assess comprehension. Review of Educational Research, 42(2), 145-170.
Anderson, R. C., Kulhavy, R. W., & Andre, T. (1971). Feedback procedures in programmed instruction. Journal of Educational Psychology, 62, 148-156.
Anderson, R. C., & Biddle, W. B. (1975). On asking people questions about what they are reading. In G. Bower, ed., The psychology of learning and motivation, 89-132. New York, NY: Academic Press.
Andre, T. (1979). Does answering higher-level questions while reading facilitate productive learning? Review of Educational Research, 49 (2), 280-318.
Angell, G. W. (1949). The effect of immediate knowledge of quiz results on final examination scores in freshman chemistry. Journal of Educational Research, 42(5), 391-394.
Bangert-Drowns, R. L., Kulik, C. C., Kulik, J. A., & Morgan, M. (1991). The instructional effect of feedback in test-like events. Review of Educational Research, 61 (2), 213-238.
Bardwell, R. (1981). Feedback: How does it function? Journal of Experimental Education, 50, 4-9.
Bormuth, J. R. (1970). On the theory of achievement test items. The University of Chicago Press.
Bormuth, J. R., Manning, J., Carr, J., & Pearson, D. (1970). Children's comprehension of between- and within-sentence syntactic structure. Journal of Educational Psychology, 61, 349-357.
Bransford, J. D. (1979). Human cognition: Learning, understanding, and remembering. Belmont, CA: Wadsworth Publishing.
Britton, B. K., & Guloz, S. (1991). Using Kintsch's computational model to improve instructional text: Effects of repairing inference calls on recall and cognitive structures. Journal of Educational Psychology, 83, 329-345.
Cantor, J. H., & Brown, J. S. (1956). An evaluation of the trainer-tester and punchboard tutor as electronics trouble-shooting training aids. Technical Report 1257-2-1, Naval Training Device Center, Orlando, Florida. (ERIC Document Reproduction Service No. ED 129 336)
Clariana, R, B. (2001). Determining the separate contributions of higher-order questions and of feedback. An unpublished manuscript.
Clariana, R, B. (1999a). CBT design: A feedback achievement treatment interaction. Twenty-first Annual Proceedings of Selected Research Presentations of the Association for Educational Communications and Technology, 22, 87-92.
Clariana, R. B. (1999b). Differential memory effects for immediate and delayed feedback: A delta rule explanation of feedback timing effects. A Presentation at the Annual Convention of the Association for Educational Communications and Technology, February, 1999 in Houston, TX. (ERIC Document Reproduction Center ED 430 550).
Clariana, R. B. (1993a). The motivational effect of advisement on attendance and achievement in computer-based instruction. Journal of Computer-Based Instruction, 20 (2), 47-51.
Clariana, R. B. (1993b). A review of multiple-try feedback in traditional and computer-based instruction. Journal of Computer-Based Instruction, 20 (3), 67-74.
Clariana, R. B. (1992). The effects of public reports of progress on rate of lesson completion in computer-based instruction. Journal of Computing in Childhood Education, 3 (2), 127-136.
Clariana, R. B., & Lee, D. (2001). Recognition and recall study tasks with feedback. Educational Technology Research and Development, under review.
Clariana, R. B., Ross, S. L., & Morrison, G. R. (1991). The effects of different feedback strategies using computer-assisted multiple-choice questions as instruction. Educational Technology Research and Development, 39 (2), 5-17.
Clariana, R. B., & Smith, L. J. (1989). The effects of AUC and KCR feedback on learners of different ability. A paper presented at the Annual meeting of the Mid-South Educational Research Association, Little Rock, Arkansas November. (ERIC Document Reproduction Service No. ED 313 387)
Clariana, R. B., & Smith, L. J. (1988). Learning style shifts in computer-assisted instruction. A paper presented at the Annual Meeting of the International Association for Computers in Education (IACE) in New Orleans, LA., April, 1988. (ERIC Document Reproduction Service: ED 295 796)
Clariana, R. B., Wagner, D., & Roher-Murphy, L. (2000). A connectionist description of feedback timing. Educational Technology Research and Development, under revision.
Collins, A., & Quillian, M. (1972). Experiments on semantic memory and language comprehension. In L. Gregg, ed., Cognition in learning and memory. New York: Wiley.
Coorough, R. P. (1991). The effects of program control, learner control, and learner control with advisement lesson control strategies on anxiety and learning from computer-assisted instruction. In Proceedings of the 13th Annual Meeting of the Association for Educational Communications and Technology, Orlando, FL.
Crocker, L. M. & Algina, J. (1986). Introduction to classical and modern test theory. New York: Holt, Rinehart and Winston.
Dempsey, J. V., & Driscoll, M. P. (1989). The effects of four methods of immediate corrective feedback on retention, discrimination error, and feedback study time in computer-based instruction. A paper presented at the Annual Meeting of the American Educational Research Association, San Francisco, CA. March.
Dick, W., & Latta, R. (1970). Comparative effects of ability and presentation mode in computer-assisted instruction and programmed instruction. Audio-Visual Communication Review, 18(3), 34-45.
Diekhoff, G. M., & Wigginton, P. (1982). Using multidimensional scaling-produced cognitive maps to facilitate the communication of structural knowledge. A paper presented at the Annual Meeting of the Southwestern Psychological Association. (ERIC Document Reproduction Service No. ED 218 245)
Dobbins, I. G., Kroll, N. E. A., Yonelinas, A. P., & Liu, Q. (2000). Distinctiveness in recognition and free recall: The role of recollection in the rejection of the familiar. Journal of Memory and Language, 38, 381-400.
Duchastel, P. C. (1983). Improving adjunct question research: Processes and ecological validity. Human Learning, 2, 1-5.
Elman, J. L. (1993). Learning and development in neural networks: The importance of starting small. Cognition, 48, 71-99.
Foltz, P. W., Kintsch, W., & Landauer, T. K. (1998). The measurement of textual coherence with latent semantic analysis. Discourse Processes, 25, 285-307.
Gilman, D. A. (1969). Comparison of several feedback methods for correcting errors by computer-assisted instruction. Journal of Educational Psychology, 60 (6), 503-508.
Glover, J. A. (1989). The "testing" phenomenon: Not gone but nearly forgotten. Journal of Educational Psychology, 81 (3), 392-399.
Godden, D. R., & Baddeley, A. D. (1975). Context-dependent memory in two natural environments: On land and underwater. British Journal of Psychology, 66, 325-331.
Godden, D. R., & Baddeley, A. D. (1980). When does context influence recognition memory. British Journal of Psychology, 71, 99-104.
Haberlandt, K. (1997). Cognitive Psychology 2nd edition. Boston, MA: Allyn and Bacon.
Hamaker, C. (1986). The effects of adjunct questions on prose learning. Review of Educational Research, 56 (2), 212-242.
Hannafin, M. J. (1984). Factors affecting the locus of instructional control in the design of computer-assisted instruction. Journal of Instructional Development, 7(3), 6-10.
Hodes, C. L. (1984). Relative effectiveness of corrective and non-corrective computer feedback on the cognitive learning of science. Unpublished Masters Thesis, the Pennsylvania State University. (ERIC Document Reporduction No. ED 311 883)
Hudgins, B.B., Dorman, J., & Harris, M. (1979). Some effects of adjunct questions on intermediate grade children with differing reading comprehension abilities. Journal of Educational Research, 72, 259-265.
Johansen, K. J., & Tennyson, R. D. (1983). Effect of adaptive advisement on perception in learner-controlled, computer-based instruction using a rule-learning task. Educational Communications and Technology Journal, 31(4), 226-236.
Jonassen, D. H. (1985). Generative learning versus mathemagenic control of text processing. In D. H. Jonassen, ed. The Technology of text : Principles for structuring, designing, and displaying text, 9-45. Englewood Cliffs, NJ: Educational Technology.
Jonassen, D. H., & Cole, P. (1993). Learner-generated vs. instructor-provided analysis of semantic relationships. In Proceedings of Selected Research and Development Presentations at the 15th Annual Convention of the Association for Educational Communications and Technology. (ERIC Document Reproduction Service No. ED 362 170)
Jonassen, D. H., & Wang, S. (1992). Acquiring structural knowledge from semantically structured hypertext. In Proceedings of Selected Research and Development Presentations at the 14th Annual Convention of the Association for Educational Communications and Technology. (ERIC Document Reproduction Service No. ED 348 000)
Kintsch, W. (2000). Metaphor comprehension: A computational theory. Psychonomic Bulletin and Review, in press.
Kintsch, W. (1994). Discourse processing. In d'Ydewalle, Eelen, & Bertelson, eds. International perspectives on psychological science: Volume 2: The state of the art, 135-155. Hove, UK: Lawrence Erlbaum Associates Ltd.
Kulhavy, R. W. (1977). Feedback in written instruction. Review of Educational Research, 47 (1), 211-232.
Kulhavy, R. W., & Anderson, R. C. (1972). Delay-retention effect with multiple-choice tests. Journal of Educational Psychology, 63 (5), 505-512.
Kulhavy, R. W., & Stock, W. A. (1989). Feedback in written instruction: The place of response certitude. Educational Psychology Review, 1, 279-308.
Kulik, J. A., & Kulik, C. C. (1988). Timing of feedback and verbal learning. Review of Educational Research, 58 (1), 79-97.
Lee, D., Smith, P. L., & Savenye, W. (1991). The effects of feedback and second try in computer-assisted instruction for a rule-learning task. Thirteenth Annual Proceedings of Selected Research presentations of the Association for Educational Communications and Technology, 13, 411-432.
Lindner, R. W., & Rickards, J. P. (1985). Questions inserted in text: Issues and implications. In D. H. Jonassen, ed. The Technology of text : Principles for structuring, designing, and displaying text, 131-157. Englewood Cliffs, NJ: Educational Technology.
Lewis, M. W., & Anderson, J. R. (1985). Discrimination of operator schemata in problem solving: Learning from examples. Cognitive Psychology, 17 (1), 26-65.
McClelland, J. L. (1981). Retrieving general and specific information from stored knowledge of specifics. In Proceedings of the Third Annual Meeting of the Cognitive Science Society, 170-172.
McLeod, P., Plunkett, K., & Rolls, E. T. (1998). Introduction to connectionist modeling of cognitive processes. Oxford, UK: Oxford University Press.
McNamara, D., Kintsch, E., Butler, S. N., & Kintsch, W. (1996). Are good texts always better? Interactions of text coherence, background knowledge, and levels of understanding in learning from text. Cognition and Instruction, 14, 1-43.
Merrill, J. (1985). Levels of questioning and forms of feedback: Instructional factors in courseware design. A presentation at the Annual Meeting of the American Educational Research Association in Chicago, Illinois. (ERIC Document Reproduction Service No. ED 266 766)
Merrill, J. (1987). Levels of questioning and forms of feedback: Instructional factors in course ware design. Journal of Computer-Based Instruction, 14 (1), 18-22.
Milheim, W. D., & Azbell, J. W. (1988). How past research on learner control can aid in the design of interactive video materials. Tenth Annual Proceedings of Selected Research Papers at the 1988 Annual Convention of the Association for Educational Communications and Technology, 10, 459-472. (ERIC Document Reproduction Service No. ED 295 652).
More, A. J. (1969). Delay of feedback and the acquisition and retention of verbal materials in the classroom. Journal of Educational Psychology, 60(5), 339-342.
Morrison, G. M., Ross, S. M., Gopalakrishnan, M. & Casey, J. (1995). The effects of feedback and incentives on achievement in computer-based instruction. Contemporary Educational Psychology, 20, 32-50.
Mory, E. H. (1996). Feedback research. In D. H. Jonassen, ed. Handbook of research for educational communications and technology, 919-956. New York: Simon & Schuster Macmillan.
Nielsen, M. C. (1990). The impact of informational feedback and a second attempt at practice questions on concept learning in computer-aided instruction. Doctoral dissertation, University of Texas at Austin.
Nishikawa, S. S. (1985). Feedback: Implications for further research and study. A paper presented at the Annual Meeting of the Association for Educational Communications and Technology, Anaheim, CA. (ERIC Document Reproduction Service No. ED 256 328)
Plunkett K., & Marchman, V. A. (1993). From rote learning to system building: Acquiring verb morphology in children and connectionist nets. Cognition, 48, 21-69.
Plunkett K., & Marchman, V. A. (1996). Learning from a connectionist model of the acquisition of the English past tense. Cognition, 61, 299-308.
Pressey, S. L. (1926). A simple apparatus which gives tests and scores - and teaches. School and Society, 23, 373-376.
Pressey, S. L. (1950). Development and appraisal of devices providing immediate automatic scoring of objective tests and concomitant self-instruction. The Journal of Psychology, 29, 417-447.
Richards, D. R. (1989). A comparison of three computer-generated feedback strategies. Eleventh Annual Proceedings of Selected Research Presentations of the Association for Educational Communications and Technology, 11, 358-367.
Rotello, C. M., & Heit, E. (1999). Two-process models of recognition memory: Evidence for recall-to-reject. Journal of Memory and Language, 40, 432-453.
Santiago, R. S., & Okey, J. R. (1992). The effects of advisement and locus of control on achievement in learner-controlled instruction. Journal of Computer-Based Instruction, 19(2), 47-53.
Schimmel, B. J. (1983). A meta-analysis of feedback to learners in computerized and programmed instruction. Paper presented at the Annual Meeting of the American Educational Research Association, Montreal, Canada. (ERIC Document Reproduction No. ED 233 708)
Schimmel, B. J. (1986). Feedback use by low-ability students in computer-based education. (Doctoral dissertation, Columbia University Teachers College). Dissertation Abstracts International, 47 (11), 4068.
Shanks, D. R. (1995). The psychology of associative learning. Cambridge, UK: Cambridge University Press.
Shanks, D. R., Pearson, S. M., & Dickinson, A. (1989).Temporal contiguity and the judgment of causality. Quarterly Journal of Experimental Psychology, 41B, 139-159.
Seidenberg, M., & McClelland, J. (1989). A distributed model of word recognition and naming. Psychological Review, 96, 523-568.
Smith, P. L. (1988). Toward a taxonomy of feedback: Content and scheduling. A paper presented at the Annual meeting of the Association for Educational Communications and Technology, New Orleans, LA.
Smith, P. L., & Ragan, T. J. (1993). Designing instructional feedback for different learning outcomes. In J. V. Dempsey & G. C. Sales, eds., Interactive instruction and feedback, 75-103. Englewood Cliffs, NJ: Educational technology.
Spock, P. A. (1987). Feedback and confidence of response for a rule-learning task using computer-assisted instruction. (Doctoral dissertation, University of Texas, Austin). Dissertation Abstracts International, 48 (5), 1109.
Tennyson, R. D. (1980). Advisement and management strategies as design variables in computer-assisted instruction. A paper presented at the Annual Meeting of the American Educational Research Association, Boston, MA. (ERIC Document Reproduction Service No. 195 246).
Tennyson, R. D., & Buttrey, T. (1980). Advisement and management strategies as design variables in computer-assisted instruction. Educational Communications and Technology Journal, 28(3), 169-176.
Travers, R. M. W., Van Wagenen, R. K., Haygood, D. H., & McCormick, M. (1964). Learning as a consequence of the learner's task involvement under different conditions of feedback. Journal of Educational Psychology, 65 (3), 167-173.
Waldrop, P. B., Justin, J. E., & Adams, T. M. (1986). A comparison of three types of feedback in a computer-assisted instruction task. Educational Technology, 26, 43-45.
Widrow, B., & Hoff, M. E. (1960). Adaptive switching circuits. 1960 IRE WESCON Convention Record (Pt. 4), 96-104.


text:
[deleted account] ( 40 moms have responded )
My daughter is half black and white, she's 3months old today and I was just curious when I can expect her hair start to change. Her hair is still relatively straight with some waves right now. It usually curls after a bath and with humidity (too cute hah :) ) My boyfriend said that his nephew (100%black) had straight hair like our daughters until he was 6mo old and then it changed, this sound about right?
- Helpful (309)
- Nice (15)
- Funny (20)
- Encouraging (30)
- Hugs (19)

subset: commoncrawl


text:
Welcome to our landscaping and building department
We are located close to the A127 and A130 which are main roads connecting London to
Southend and Rayleigh to Chelmsford and the A12.
We are open 7 days a week and hold a huge range of landscaping and building
products at competitive prices.
The range of natural stone we display and stock is unrivalled locally.
Complemented by many other products that you would expect from a builders merchant,
such as: bricks, blocks, cement, fencing, tools, fixings, concrete paving, block paving,
tarmac, turf, topsoil, compost, bark, railway sleepers, building aggregate, decorative stone
and cobbles, rockery, sand, drainage, weed matting, sealants and more.
We have 12 x 40 tonne bays where our customers can collect loose building aggregate
and landscaping barks at very competitive rates.
We do of course offer loose loads and bulk bags delivered for those that cannot collect.


text:
NEMEET is a collaboration and productivity platform for commercial real-estate professionals. Basic access is free!
If you work in, or closely with, the commercial real-estate industry, you qualify for access.
Oops! We're not quite sure what happened. Press "Reload" to reload the page. If the problem persists, please contact us:
Show error details
The error occured during a -request from . The request returned the HTTP-response code
Show request response


text:
The Street Address Is:
1381 Grand Bay Wilmer Road North
Mobile, AL 36608
NOTE: Sunnyland Satsuma is located closest to the intersection of Grand Bay Wilmer Road North and Tanner Williams. Be aware that the street numbers on Grand Bay Wilmer Rd are not consecutive.
To Find Sunnyland Satsuma in Tanner Williams AL follow directions below:
From Mobile, Alabama:
From Mississippi Gulf Coast:


text:
Tourist's first North American tour found him hitting a CMJ showcase thrown by our good friends over at Pigeons & Planes. If you didn't catch him then, don't fret; he's set to hit our side of the Atlantic starting on March 10, where he'll be hitting Output for a special Annie Mac Presents show in Brooklyn. From there it looks like he's touching down in Austin, Texas for a number of dates during SXSW. He's also set to hit the Ultra Music Festival at the end of the month, with stops at Counterpoint, Sasquatch, and Movement (among others) during his summer in the U.S., with loads of dates to come. We fell in love with his latest, "I Can't Keep Up," and hope that he'll be bringing that and more jams along the way. Check out the full list of dates below.
The post Tourist's North American Tour Kicks Off On March 10 appeared first on Do Androids Dance.
Source: DAD


text:
Moderator: GOLDEN-ARM
Tadman wrote:Absolutely. Scuttlebut is that all repaints will get this scheme. Looks like a pricey paintjob compared to blue dip. That said, the railfan in me is glad to see them quit using the ARR scheme or the atrocious RailAmerica scheme. If they wanted to save money, use the blue dip. If not, do red/yellow.
(according to another picture of 714 in new paint at the same website).
Tadman wrote:Here's what I don't get - when they ordered those SD70's a few years ago, and they had so little imagination as to completely borrow the Alaska RR paint job, they could have substituted red for the blue and had a perfect modern FEC look. Red/yellow/red. Very little thought or effort required.
Return to Florida East Coast Railway
Users browsing this forum: No registered users and 2 guests


text:
MenTalkisms is a collection of 800 quotes, thoughts and experiences of men I've come across in my life as well as some of my own. No one famous. All anonymous. The quotes in this book were chosen because they are clever, offbeat and generally shorter. While different than the others in series, MenTalkisms is still intense, raw, very funny and mildly confusing.
Not everyone loves every quote, but there is something for everyone in these books and they are guaranteed to have you laughing out loud, grossed out, pissed off, thinking deeply, touched emotionally or stimulated sexually.
From the Men Talk book series.


text:
After Crushing and Comminution, add the gold ore slurry into 9 Leaching Agitation Tanks, pour in cyanide solution in the first 2 tanks and Activated Carbon in the following 7 tanks for the reverse-absorption process--This design shortens the time period of gold staying in the cyanide pregnant solution and recovers gold in advance, which sees a faster investment return compared to other designs.
Filter Press and Vibrating Screen is needed in gold ore slurry reverse-transmission process.
Xinhai has a design that ensures the minimum carbon gold loses during transmission. It lower the gold loses and the processing line is easy to maintain.
After the carbon leaching and Elution of gold, NaCN and NaOH solution is used for removing the gold from loaded carbon at a high temperature. Then the loaded solution passes through the electro winning cell and the metals are attached to cathodes.
Xinhai Desorption Electrolysis System can successfully aggregate these steps in one processing unit, the pure gold and other useful metals are highly attained via this system.
The latest CIL Plant Xinhai has constructed locates in Tanzania, Xinhai brought up a satisfying 1200tpd project with the mine owner from Tanzania Bulsowa Mining Co., Ltd.
Bringing mine owners LARGE profits, Xinhai has set up over 50+ gold CIL plants worldwide and build a reputation in the field of mineral processing technology since 1997. Xinhai provides Whole Line Plant & Design Personnel Training, helping mine owners with Equipment Procurement and Construction from ore to pure gold.


text:
Hi y'all...back from a couple days away....
full info at:
Systems Affected
Web servers running PHP
Multiple vulnerabilities exist in the PHP scripting language. These vulnerabilities could allow a remote attacker to execute arbitrary code with the privileges of the PHP process.
I. Description
PHP is a scripting language widely used in web development. PHP can be installed on a variety of web servers, including Apache, IIS, Caudium, Netscape and iPlanet, OmniHTTPd and others. Vulnerabilities in the php_mime_split function may allow an intruder to execute arbitrary code with the privileges of the web server. For additional details, see


text:
What is the weather like in Hobart?
Hobart experiences four distinct seasons, each with its own beauty and charm. Plan ahead with this information on temperature and rainfall.
Summer (December – February)
During summer in Hobart, average temperatures range from 11.5 - 21°C (52.7 - 69.8°F). The months of January and February are also the driest, and beaches are warm enough to swim.
Autumn (March – May)
Autumn has many calm, sunny days, with average temperatures between 8.9 - 17.3°C (48 - 63.1°F). The city is blanketed in deep red and yellow hues as the trees begin to change.
Winter (June – August)
During winter the island feels the southerly winds from the Antarctic and experiences some of Australia's coolest temperatures. Mount Wellington is dotted with snow and average daily temperatures range from 5 - 12.3°C (41 - 54.1°F). Overnight temperatures can drop as low as 3°C (37.4°F).
Spring (September - November)
During spring, average temperatures range from 7.8 - 16.9°C (46 - 62.4°F). This is the wettest time of year (168.7mm / 6.6″), however it should be noted that Hobart is Australia's second-driest capital city (after Adelaide).
Find out more about the weather in Hobart
When does daylight savings occur in Hobart?
Daylight saving begins at 2am, Eastern Standard Time, on the first Sunday in October and ends at 3am Eastern Daylight Saving Time on the first Sunday in April. During this time Hobart moves to Australian Eastern Daylight Time (AEDT), UTC +11.
What time is sunrise and sunset in Hobart?
December experiences the longest hours of daylight with sunrise at approximately 5:30am and sunset at 8:45pm. June has the least hours of daylight with sunrise at approximately 7:40am and sunset at 4:45pm.
What is the sea temperature in Hobart?
In summer, average sea temperatures in Hobart range from 15.6 – 16.5°C (60 - 61.7°F).
Does it snow during winter in Hobart?
Snow rarely settles at sea level, however Mount Wellington in Hobart experiences sporadic snowfall during the winter months, and occasionally through autumn and spring. Skiers and snowboarders can get their thrills in the Mount Field National Park, a 90-minute drive from Hobart.
How should I dress for winter in Hobart?
Winter days are usually clear and crisp, and indoors, many restaurants and bars are heated with an open fire. Pack a warm, rainproof jacket and dress in layers.
How should I dress for summer in Hobart?
During the day light clothing is enough, however the temperature can drop in the evenings so bring a light jacket so you can stay comfortable.
What is the Ultraviolet (UV) Index Forecast?
Visitors to Hobart should look out for the UV index in local weather reports, which describes the daily solar UV radiation intensity. Protect yourself from sunburn by using SPF30+ sunscreen, which is readily available throughout Australia at supermarkets, convenience stores and pharmacies, and wearing sun-smart clothing, including a hat and sunglasses.


text:
Search results for: Hydraulic Hose
Hydraulic Hose
Modular Elevator Manufacturing, Inc (Rating: 1) , Votes:Builds and installs custom commercial modular elevators including hydraulic, MRL elevators & prefabricated hoistways.http://www.modularelevator.netCategory: Directory/Business/B2B/Construction
Cutter Sales (Rating: 1) , Votes:Distributor and supplier for industrial bearings, ball bearings, sprockets, v-belts, chains, pulleys and hydraulic hoses plus fittings.http://www.cutter-sales.comCategory: Directory/Business/B2B
Mystik Lubricants (Rating: 0) , Votes:Offers a wide selection of performance lubricants, such as synthetic and heavy duty motor oils, grease, and hydraulic fluids, to customers online.https://www.mystiklubes.comCategory: Directory/Shopping/Automotive
Oil Flush - Oil Flushing (Rating: 0) , Votes:Reliability solutions Offers oil flushing, hydraulic system flushing and chemical cleaning soutions that offer full decontamination of equipment.http://www.rsi-usa.com/Category: Directory/Business/Industry
Metal Fabrication Equipment (Rating: 1) , Votes:Hellerson offers high quality metal fabricating equipment including press brakes, angle rolls, and hydraulic shears.http://www.hellerson.comCategory: Directory/Business/B2B
Tool Price (Rating: 5) , Votes:Nationwide supplier of quality hand and power tools at discount prices. Includes automotive tools, woodworking tools, and accessories.http://www.toolprice.com/Category: Directory/Shopping/Automotive
Pallet Trucks & Pump Trucks (Rating: 1) , Votes:Specialist in pump trucks and pallet trucks since 1984, we import pallet trucks and lift tables directly from the manufacturer.http://www.MidlandPalletTrucks.comCategory: Directory/Business/Industry
Related searches:
Webotopia provides a business directory designed to promote Hydraulic Hose companies, products and services whilst helping it's users find the Hydraulic Hose information they need. Webotopia business directory, with the help of it's dedicated support specialists and reviewers, enables delivery of Hydraulic Hose quality content, excellent customer support as well as value added services such as content publishing and press releases for it's users.


text:
Visions of heaven
Francis Gardomasks us to consider seriously what will happen to us after death and to build up clearer pictures of life after life
When one arrives to take a funeral at the crematorium, the chances are that the previous one will still be in progress - either because it started late, but more probably because the minister who is officiating is taking his or her time, cataloguing the virtues of the dear departed or speculating about the heavenly reunion which, supposedly, is taking place between the late lamented and their pre-deceased spouse.
It is instructive to listen-in to these proceedings because they usually bear no relation whatsoever to what Christians, Jews, Muslims - or Jehovah's Witnesses for that matter - have ever believed: and my guess is that, if asked to justify what they have said, their answer would be that it is 'the sort of thing that bereaved folk want to hear'.
Avoiding the truth
There's a word for 'telling people what they want to hear' and that word is 'flattery'. And flattery, they tell you, gets you nowhere.
Be that as it may, one might have supposed that when people are confronted by something as serious and inevitable as Death, whether their own or someone else's, they would at least want to be told the truth.
Seemingly this is not so. Over and over again people claim that they are 'not interested' in what happens to them beyond the grave.
This may sound rather high-minded, even praiseworthy. Not to be concerned for one's welfare after death sounds a lofty ideal. But it takes on a different aspect once you ask such a person exactly whose future life they are not interested in. Can they really say, standing by the death-bed of someone they have loved, 'You are about to die, and frankly I am not interested in what happens to you after that'? The simple transference from thinking about their own death to that of someone else changes the whole picture, does it not?
Does it not begin to look as if the question of what happens when we die (especially as it applies to other people) is one which cries out to be addressed. It maybe the answer we come up with, after due thought, is 'I simply don't know,' but that answer is light-years away from saying 'I'm not interested.'
No direct experience
It is true that being alive we can have no first-hand experience of what being dead is like, and the great world religions have constantly warned their disciples against becoming too curious about the subject. But this is different from saying that it's just a matter of blind guesswork, in which one person's ideas are as good as another's. Christians believe that God has given them enough data to build up some sort of picture of what he has intended, though much of it has to be expressed in picture-language and by analogy.
So, much of our speculation about life hereafter depends precisely upon the use of images. That is why we have to remember that images have different effects upon different people. To some, not all, listening to music is the experience which puts them most closely in touch with the immaterial: so, not surprisingly, some people imagine that there will be a dimension about the life everlasting that, in some respects, resembles music. For others, it may be the spoken word. For yet others, natural beauty resonates of the eternal, whilst yet others still, find the sheer vast-ness of space, the solar system, and terrestrial mountains speaks to them of Gods majesty.
Of course, we must not make the mistake of equating any of these things with their Creator. Earth, sky, sea, sun, stars and nature, music and art - none of these is God. But Christians have always insisted that in Jesus Christ the Word of God was made flesh and dwelt among us. Christianity is, therefore, an incarnational faith and nothing material or fleshly can, in itself, be bad or evil. Though the world and the flesh may lead us astray, they are not in themselves things that are evil. Hence there must be a continuum between what is obviously good in this world and what we shall find available in some form in the next. If this were not so, then it would mean that God was prepared to destroy, quite wantonly, the good in things he has created.
Now that's not at all to say that we shall discover what is innocent and enjoyable in this life precisely replicated; but it does suggest that part of our post-mortal experience will be that of being enabled to enjoy to the full what we have been able to enjoy only in a partial way on earth- The partial and transitory nature of earths pleasures are part of the scheme of God to encourage us (to tempt us, if you like) to prepare ourselves for the vision of himself, which is the End for which he created us. As the Shorter Catechism puts it, "The Chief End of Man is to glorify God and enjoy him for ever.'
Heaven and hell
This presupposes that in our pursuit of what is enjoyable on earth we have not in the meanwhile so defaced his image in ourselves that we become incapable of enjoying anything permanently. If you think about it, any worldly pleasure, if sought to the exclusion of all other considerations, ceases to give pleasure sooner or later and becomes an evil. Gluttony and alcoholism are but two examples. The real risk to our eternal future is not that God will refuse to let us into heaven, but that there will be nothing left within us capable of enjoying it. The final divide is between those of us who have said to God 'Thy will be done (in us)' and those to whom God says 'Your will be done (in yourselves)'. The gates of hell are locked on the inside, and its inhabitants, by refusing the Good when they see it, prefer to keep their own company with other like-minded people, or perhaps in perpetual solitude.
The keynote of heaven is fulfilment by letting ourselves, of our own free will, become conformed with what God designed us to be. The keynote of hell is boredom and frustration because in his grace he will allow us, by using the free will he has given us, to employ it in separating ourselves as widely as possible from him.
The consequence of this may well be that those who refuse God's mercy will undergo a process comparable with shrinkage. In the eyes of the Redeemed they will shrink into insignificance. In their own they may well appear to be of supreme importance as individuals. The problem will then be that everyone around them will be thinking precisely the same thing about themselves.
What a bore!
Return to Home Page of This Issue
Return to Trushare Home Page


text:
Karen is an athletic, no-nonsense instructor extremely
well regarded in the fitness industry. You could call her
the antithesis of Denise Austin--there's
not a "cutesy" bone in her body, so her
routines appeal to men as well as women. She has so
little body fat that you can see practically every muscle
on her body.
Karen uses superb cuing and impeccable form to teach a
huge variety of large limb movements that work all major
muscle groups through their full range of motion. Lots of
level changes, directional changes, and balancing moves
keep her choreography interesting and intense. Karen's
movements are graceful and beautiful to watch. Karen's
tapes are well thought out so you get an incredibly
thorough workout that leaves your body feeling balanced
and refreshed.
My favorite of her tapes is "Energy Sprint," which has
received terrific reviews in major fitness and consumer
magazines. It embodies all that is good about Karen's
workouts: excellent instruction, lots of intensity
modifications and form pointers, and choreography that
burns lots of calories, builds muscle and increases
cardiovascular endurance.
Wendy Niemi
Karen is one of the most highly-praised fitness
instructors in the business, and possesses a sleek,
sculpted body that would make anyone want to get with
whatever program she's on. Her style is most often
described as "no-nonsense" - she gets right
down to brass tacks as if she's right there in your home
training you. While Karen never gets "chatty",
her instruction is perhaps the most thorough, with
detailed explanations of exactly how to place your
weight, etc. Karen has a knack for coming up with a wide
variety of ways to work your muscles, even
often-neglected ones like your shins, and she
choreographs them so as to make her toning routines
interesting and dance-like, unlike so many muscle-toning
videos that are little more than boring, repetitive
calisthenics drills. As far as I know, Karen's videos are
all for intermediate to advanced exercisers. It would be
great if she could produce a video or two for beginners,
as they could benefit a great deal from her expertise.
But no one who's serious about exercise should be without
a Voight video or two on their shelf.
Sue Bryant
The best instructor! Clear, easy-to-understand, no
extraneous chatter, and the fittest body I have ever
Robin Mordfin
Karen Voight is absolutely my favorite instructor.
She's a welcome relief from all the 'hooting and
hollering' instructors out there. Both her aerobics and
toning segments in all videos are obviously
well-thought-out, very precise and definite -- while
being fun at the same time. I'm always confident that, if
it's included in a Voight video, it must be good for me!
She's the perfect role model!
Joni O
What can one say about Karen Voight that hasn't
already been said? She's got everything -- outstanding
instruction, cuing, form, and choreography. Karen is a
real pro. I don't think you can fault her on anything
when it comes to teaching.
Annie S.
Karen's matter-of-fact instruction is very refreshing.
I thoroughly enjoy her enthusiasm which is inspiring, but
not annoying. Her cuing is excellent.
Personal trainer to Paula Abdul; served as fitness
consultant for Abdul's "Get Up and Dance"
workout video
Even a fun-loving "party animal" like me can
find a lot to enjoy about Karen Voight. She's a
consummate professional who is serious about producing
thorough, effective, intelligent, and--yes--fun workouts.
It's wonderful to see someone of her calibre getting the
national press coverage she deserves--in 1996, she's been
quoted as a fitness expert in many women's magazines, and
in May '96, US News and World Report put together a
wonderful spread on her that's available on-line at their
website. While she's more reserved than, say, Donna
Richardson, Kari Anderson, and Cathe Friedrich, her
sincerity, warmth, and sensibility shine through both in
her tapes and the interviews she's conducted for the
media. Karen has both the physique and the personality to
inspire confidence in her programs. All those celebrities
who come to see her can't be wrong!
Melissa Cooper
Aside from excellent cuing and being the
"Instructor's Instructor," Karen Voight has the
body that any woman would die for, and I am no exception!
I like her "get down to business personality"
without having a "hard and rough" edge. Her
transition from aerobic to endurance makes me feel like
I'm right there with the group. She is serious enough
without being too chatty! I love her concentration and
instruction on form. S&SM is my first Voight tape. I
know that she will become a staple in my collection that
won't collect dust.
My all-time favorite instructor, Karen's form is
impeccable her cuing is perfect and she adds so much
value with her tips on form and proper execution.
I find Karen so likeable and motivating. Her
instructions are always so clear and she gives you tips
on common errors to avoid, good images to help you do the
exercises correctly.
Amy Pfau
I love Karen as an instructor. She is one of my
favorites. Her routines are not boring because they are
different. Because her routines really seem to focus on
the muscle that's being worked, I really see and feel the
results quickly. The clever ways designs her workouts
makes me feel like I'm not working to the point of
exhaustion, like some videos, yet my body really feels
the like it's used muscles in a way that they've never
been used before.
Lynette Vessels
I have to say a few words about Karen Voight. She has
an excellent, well scuplted body that any woman would die
for. Of any instructor, I would prefer to look like she
does, however, I find her instructing to be the most
annoying of anyone on the market (with the exception of
anything done by Denise Austin, some things by Tami Lee
Webb or Donna Richardson). What is this "IN, OUT,
UP, DOWN" business????!!!! I'm so annoyed by it I
want to shut off the sound - this constant instruction
(IN OUT UP DOWN) combined with vocal music drives me
insane. I don't know how anyone can stand her cueing.
I own every Karen Voight tape made and I believe that
"Energy Sprint" is the best tape she has made.
In this video she doesn't do much of the cueing that I
find so offensive. For being in such marvelous shape, I
can't understand why most of her other videos aren't as
tough. Strong and Smooth moves is a decent workout but
not enough of a challenge for an advanced exerciser.
Denise Zoldos
Karen Voight is my favorite instructor. Although I
like the FIRM tapes better for their intensity, as an
instructor Karen is one of the best. I like her matter of
fact style, her excellent cues, and her concern for the
exerciser. I really believes she wants me to be fit! :)
Katie Reaves
Karen Voight is the best instructor to watch to learn
the correct form for exercises. She is not perky and
comes off as shy in her opening words.
Abby N
Maybe it's the hairstyle and color, or the shape of her face, but on Energy Sprint, Karen resembles a toned, buff version of singer Helen Reddy (who sang the great anthem "I Am Woman"). True, her "down-up, down-up" cuing can be annoying, but on a tough workout like this where good form is imperative, I'd rather have too much than not enough instruction. Being a musician myself, I also appreciate her excellent taste in music both on ES and Strong and Smooth Moves. Karen has always come across as very intelligent and professional, and I can't imagine her using disco-type mixes in her workouts. Instead, what she selects is very jazzy and a joy to listen to:)
Melissa Cooper
I think my favorite instructors are people I would choose as friends (Cathe Friedrich, Kathy Smith, Susan Harris, Tracey Long ...). Karen Voight is not someone I would pick as a friend - she is too serious for me, and does not seem to have a great deal of humor. Descriptions of her eating habits on the VF forum make it clear that she and I could never go for dinner together - I love ice cream :-) I realize some people like the serious tone of Karen's videos, but if I am going to spend an hour or more of my day working out with someone, I want to have fun. That said, I do respect Karen Voight's knowledge, and her workouts are safe.
I think Karen Voight is one of the most uneven instructors around. I think her Energy Sprint and Great Weighted Workout are among the best fitness tapes around, but I couldn't stand Strong and Smooth moves.
I will not buy Streamline Fitness because I have now developed such a dislike for Karen's attitude about the fitness industry.
First of all, she tells Collage Video that she will only market her video mail order because she thinks fitness industry is too commercial. But she's now had very obvious breast augmentation surgery-- I mean, what a hypocrite! How commercial can you get! Her new bustline is quite obvious in the Streamline Fitness ad and in the print ads for exercise wear.
I'm sure that a lot of the instructors on video tapes I own have probably enhanced themselves surgically, but at least they don't make any pretensions of being above the industry.
In my opinion, it doesn't get any better than a Karen Voight workout. I have been working out for years and have more tapes than I know what to do with. I've tried out all of the ones with the models, but none have kept me more interested, inspired and in pain! The only thing that annoys me is that she had to do a tape with Elle McPherson. Don't get me wrong, I love the workout, but I could do without Elle. The set is magnificently beautiful and I try to imagine that I'm there instead of my living room. I would prefer not to use a step, but with Karen I don't mind. I have yet to purchase her equipment such as her body bar and her weighted ball. She's never boring and each time I use her tapes, I find I'm more conscious of my form and not just trying to get it over with. I can't stand Denise Austin or Kathy Smith or those Buns Of Steel instructors. This tape is for people who want to be serious about their fitness routine and not in to lose those last couple of pounds. I know she is the one who started the "spinning" craze, and I've seen the bike for an astronomical price. But I'd like to know if anyone has a video that I could use with my own bike. If anyone has used it, please post the message here. I'm interested to see how it is. Also, I'm glad I didn't purchase the Tae Bo tapes. I really wanted to, just for some variety, but after hearing about the miscounts, disco effects and insufficient warm-ups, you guys saved me a bundle and the hassle of a return! Thanks a lot!