{'fraction_non_alphanumeric': 0.0217588769611891, 'fraction_numerical': 0.0027663088356729977, 'mean_word_length': 5.6632737276478675, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Seeing the real world: Comparing learning from enhanced lecture demonstrations and verification labs

Instructors in introductory physics courses often use labs and demonstrations to reinforce that the physics introduced in lectures and textbooks describes what actually happens in the real world. The surface features of these activities are similar and instructional learning goals are usually identical, but physics education research has surprisingly found very different impacts of these instructional methods on students' physics content knowledge. In this paper we discuss possible mechanisms for the measurably different learning outcomes by dissecting the activities by components that are known to influence student learning and engagement: the role of prediction, assessment, cognitive load, and engagement. Identifying mechanisms for students' learning in an instructional method is critical to understanding the results of its impact and for its successful implementation.

Instructors in introductory physics courses often use labs and demonstrations to reinforce that the physics introduced in lectures and textbooks describes what actually happens in the real world. The surface features of these activities are similar and instructional learning goals are usually identical, but physics education research has surprisingly found very different impacts of these instructional methods on students' physics content knowledge. In this paper we discuss possible mechanisms for the measurably different learning outcomes by dissecting the activities by components that are known to influence student learning and engagement: the role of prediction, assessment, cognitive load, and engagement. Identifying mechanisms for students' learning in an instructional method is critical to understanding the results of its impact and for its successful implementation.

I. INTRODUCTION
In this paper, we discuss the similarities and differences between two instructional methods designed to teach physics concepts: enhanced lecture demonstrations and verification labs. Recently published research has found that labs do not add to students' learning of physics content in a measurable way. 1,2 These results may seem puzzling when compared to enhanced lecture demonstrations, which share common surface features with labs and have been shown to improve student learning. 3,4 In both activities, students must predict outcomes, observe the physics, reflect on their observations, and converge on a final explanation or concept. They both involve interacting with peers and provide experiences that may be deliberately counterintuitive but eventually demonstrate that "physics works." The stark contrast between these research outcomes (substantial learning gains in one context but not the other) raises questions: why are these structurally similar activities yielding such different learning outcomes? In this paper, we compare labs to lecture demonstrations to decompose when and why these instructional methods do and do not impact learning of physics material.

A. Definitions
Many introductory physics laboratories, such as those discussed in the studies above, 1,2 use hands-on activities as a way to reinforce physics content and to demonstrate that physics applies to a real-world setting. These labs consist of well-defined, structured experimental protocols that aim for students to verify physics ideas introduced in the lecture portion of the course. In these labs, students engage in structured sequences of predictions, observations, and explanations. We refer to these labs as verification labs.
Enhanced lecture demonstrations are structurally similar to verification labs. Students make a prediction about the outcome, observe the demonstration, and then may explain-to themselves or to others-the physics that explains the observation. Interactive Lecture Demonstrations are a specific type of enhanced lecture demonstration that includes eight steps: a description of the demonstration, an individual prediction, student discussion about their predictions, a class discussion about students' predictions, a revised individual prediction, the demonstration, a class discussion and individual writing about the results, and a class discussion of analogous physical situations. 5 Enhanced lecture demonstrations do not necessarily include all components of Interactive Lecture Demonstrations, such as peer discussion. Enhanced and Interactive lecture demonstrations contrast with traditional lecture demonstrations where students are positioned to be observers of the demonstration with no requirement to generate or record predictions or explanations or discuss with peers.
In what follows, we evaluate verification labs and enhanced lecture demonstrations according to the role of prediction, assessment, cognitive load, and student engagement (summarized in Figure 1). We argue that such dissection of an instructional method in terms of the mechanisms for learning is critical to understanding the results of its impact and for successful implementations or adaptations of that instructional method. 6 While we discuss verification labs, we do not intend to generalize to all types of labs. Many labs aim to teach lab skills and the nature of science, rather than physics content, and so this discussion does not apply to those labs. Throughout this paper, we also distinguish between traditional, enhanced, and Interactive lecture demonstrations, outlining how the differences between them relate to the measured learning outcomes. Many of the reasons that we claim student learning from verification labs is limited will also apply to why learning from traditional demonstrations is limited.

II. PREDICTIONS
The generation of a prediction is said to be the critical component to measured gains in students' content knowledge from enhanced lecture demonstrations. 3,4 Both verification labs and enhanced lecture demonstrations ask students to make predictions using a number of representations (mathematical, conceptual, etc.). The timing and ways students develop and confront predictions in enhanced lecture demonstrations may be a structurally different process, however, from developing and confronting predictions in verification labs.
Enhanced lecture demonstrations require students to make an initial prediction about an overall result using their real-world experiences, their intuition, prior physics instruction, or their gut-whatever resources they decide to draw on. They then must commit to their prediction by recording it, either through an open-response, closed-response, or combination-response format. 3 Open-response formats could include algebraic representations, drawing a picture, sketching a graph, or writing a description. Closed-response formats often involve selecting from a set of multiplechoice options through a "clicker-question" vote. As a combination format, students may first develop their predictions in an open-response format, and subsequently be shown multiple-choice options to vote on, typically based on common relevant ideas. Through the clicker-question vote, students must consider a set of deliberately chosen contrasting cases in evaluating their own reasoning. 7 Experiencing the demonstration provides feedback so that students must confront their recorded ideas, providing opportunities for them to make modifications, if appropriate.
Engaging students with a prediction before observing and explaining the demonstration also serves to initiate or activate an organizational structure for their knowledge, 8 creating a so-called "Time for Telling." 9 In Interactive Lecture Demonstrations, a critical step after discussing the results of the demonstration is to discuss analogous physical situations so that students can further build up and generalize their knowledge organization. 5 Verification labs use predictions in a different way: students most often apply previously learned knowledge and equations to form predictions about specific values or outcomes. Students are rarely, if ever, expected to contribute their individual knowledge or ideas in generating their predictions. Students are often guided to use particular representations that are deemed appropriate by the instructor, often equations or graphs using specified parameters. For example, if a lab is an investigation of the motion of a cart on a ramp, students may be prompted to draw a free body diagram for the cart, produce a corresponding algebraic expression of the cart's motion, and use that expression to predict the value of parameters to be measured.
The purpose of the lab experiment is generally to verify that the physics is "true" in the real world. The predicted values, therefore, come from authoritative resources-lecture, teaching assistants, or the textbook-rather than from students' individual ideas. Within the verification lab experience, therefore, students may never directly confront their own incorrect or novice-like ideas.
Another important consideration is the timing of the prediction. In verification labs, the predictions are most often made after instruction on the topic, so that predictions are applications of previously learned ideas. A demonstration, however, may be shown to students before or after instruction on the topic. It is unclear, therefore, whether the timing of the activity (and the prediction) is important for understanding the impacts on student learning.

III. ASSESSMENT
Assessment influences what students do, how they act, and to what they pay attention. In enhanced lecture demonstrations and verification labs, different assessment strategies may encourage students to focus on certain aspects of the activities.
In verification labs, students may be assessed on lab notebooks, lab reports, completing worksheets, or participation. Often, credit is given for correctness (either implicitly or explicitly): correctly following the provided methods and procedures, carrying out calculations correctly, and generating correct explanations. Having the correct answer in these labs does not necessarily mean that students developed the correct content knowledge or understanding from the lab activity.
In contrast, enhanced lecture demonstrations are most often assessed for participation, which includes various combinations of generating a prediction, recording or interpreting data and observations, and explaining the results of the demonstration. Most steps are student-generated and students are able to record any prediction, regardless of correctness, usually without incurring penalty. Often, the instructor uses student input to generate the correct explanation. Assessment, therefore, comes from student participation, with clear messaging that the focus is on student thinking, not correctness.
Even when both activities may be assessed on participation, the nature of that participation differs in two significant ways. Firstly, student participation from enhanced lecture demonstrations is required for the activity to progress, whereas in verification labs, student participation is not built on or discussed. Secondly, verification labs give students an optimal procedure to follow, so their participation is motivated towards correctly carrying out (performing) the procedure, rather than mastering the content. 10 In enhanced lecture demonstrations, however, the focus is on developing ideas in a mastery, growth-mindset way: 11 it's alright if we make a wrong prediction, as long as we can make sense of it afterwards.
Furthermore, verification labs rarely include structured opportunities for students to self-revise, so the stakes are higher when assessed on correctness. In-lab discussions with instructors may provide students with opportunities for revision, but assessing on correctness may encourage students to view the instructor as a source of confirmatory feedback. In enhanced lecture demonstrations, in contrast, the balance of low stakes assessment with immediate feedback and opportunities for revision encourages participation and self-revision of incorrect predictions.
With the focus on student thinking rather than correctness in enhanced lecture demonstrations, there is still incentive for students to generate correct predictions. Enhanced lecture demonstrations are used to draw out students' "misconceptions" through predictions and to provide students with immediate, real-world feedback to revise predictions and refute misconceptions. This motivation is probably obvious to students, so making a correct prediction becomes a source of pride and implicit incentive for participation. In Interactive Lecture Demonstrations, students are accountable to their peers, providing further incentive for thorough reasoning.

IV. COGNITIVE LOAD
Cognitive load refers to the notion that an individual has a capacity (i.e., limit) for processing information in their short-term memory. 12 It is supported by research suggesting that individuals can only store so much in their memory at once or can only pay attention to so many things at a time. The way in which information is presented to learners, the way learners must interact with information, or the sheer amount of information presented to learners can impact their cognitive load. 13 Simplistically, high cognitive load impedes learning.
In any real-world, hands-on activity-be it a lab or a demonstration-there are a number of features that contribute to the user's cognitive load. In a verification lab, students must coordinate equipment, physics theory, equations, concepts, data and its variability, a notebook or worksheet, and the complexities of team work, among other tasks. In enhanced lecture demonstrations, much of that work is done for the students. Their main cognitive task is related to making predictions and coordinating their predictions with their observations. In this way, we argue that a verification lab is a high cognitive load activity, while an enhanced lecture demonstration is a much lower cognitive load activity.
Other research, however, has argued that high cognitive load is not necessarily a bad thing. 7,14 In those, and other studies, ill-structured problems with arguably high cognitive load were found to improve student learning over highly structured activities with lower cognitive load. These studies typically refer to high cognitive load in the context of large degrees of freedom for student attention and exploration, but those degrees of freedom still focus student attention. Part of the argument is that students' attention in low-cognitive-load situations is overly constrained such that students do not need to activate their prior knowledge. 14 In verification labs, the higher cognitive load primarily refers to a large number of objects, concepts, and procedures to be manipulating and coordinating at once that contribute little (if at all) to the learning objectives. There are very few degrees of cognitive freedom related to the learning objectives.
This discussion can also apply to distinguishing traditional from enhanced lecture demonstrations. Depending on the equipment, a demonstration can involve a huge number of variables or components, as described above. Traditional lecture demonstrations leave many of those variables unstructured, with many degrees of freedom that do and do not relate to what is to be learned. The structure of enhanced and Interactive lecture demonstrations serve to focus student attention towards the desired cognitive degrees of freedom. The RealTime Physics materials are great examples of how physical demonstrations can be designed to focus student attention towards the intended learning objectives, removing unnecessary cognitive load. 15

V. STUDENT ENGAGEMENT
The ways that students engage in learning activities have been characterized by students' behaviors and hierarchically ordered based on learning outcomes: 16 interactive > constructive > active > passive.
In passive activities students are, well, passive. In active activities students participate in a physical action such as manipulating objects, gesturing, repeating, or highlighting; the physical actions activate students' existing knowledge. Constructive activities require students to generate new knowledge, which includes activities such as explaining a concept, generating predictions, and making connections. Students engaging in interactive activities use a partner's (e.g., instructor, peer, computer tutor) contributions to build new knowledge, which includes activities such as revising from feedback and arguing. This interaction also serves to provide the individual with feedback on their own knowledge. By these definitions, each category contains the previous categories, so interaction requires construction (not just interacting with another person). Interactive activities are thought to produce greater learning gains than constructive activities, because they use shared ideas to construct new knowledge from multiple sources and perspectives. 16 Because students may observe a demonstration while thinking about the relevant physics, we conservatively categorize traditional lecture demonstrations as active activities. In practice, students may view traditional lecture demonstrations merely as entertainment, a passive activity. 17 As we've discussed already, there are significant, measurable differences in learning gains between traditional and enhanced lecture demonstrations. 3,5 Enhanced lecture demonstrations are interactive because students receive feedback (or new knowledge) from observing the demonstration and, as a result, are able to revise their understanding of the situation. Interactive Lecture Demonstrations incorporate additional interactive elements such as redeveloping (or maintaining) a prediction following peers' contributions. Often, students discuss again with their peers after observing the demonstration to generate an explanation before receiving instruction. Interestingly, the difference between enhanced and Interactive lecture demonstrations are statistically small or, perhaps, insignificant. 3 As defined, verification labs certainly meet the criteria for active activities. Students use their hands to work with equipment, answer questions, and often manipulate data and equations. Generally, the intent is for labs to also be constructive or interactive; students work in pairs or groups (suggesting interactivity) and have opportunities to construct explanations of the data according to relevant physics concepts. Intent and opportunities for construction and interactivity, however, do not necessarily translate into action (and the measured differences between traditional and enhanced lecture demonstrations is a clear example of this). In verification labs, the connections between the experimental apparatus and mathematical formulation are typically detailed in the instructions. Reflection questions or questions requesting explanations more often require application of previously learned concepts or procedures, rather than generation of new ones. Without construction of new knowledge, there is no interactivity (again, interactive does not just mean talking to someone else in this context).

VI. DISCUSSION
In this paper we have outlined possible mechanisms for the differences in learning outcomes between verification labs and enhanced lecture demonstrations. In enhanced lecture demonstrations students generate an individual prediction, are assessed for their participation and mastery of content, are able to focus attention on key aspects, and engage in interactive learning. In verification labs students base their predictions on authoritative sources, are assessed on their correctness or performance, have many experimental features they must attend to, and do not engage in constructive or interactive learning.

A. Limitations
Given the lack of existing research comparing these two instructional modes, there are inherent weaknesses in our arguments. The role of the prediction in enhanced lecture demonstrations is to induce conceptual change: 18 Students confront their incorrect ideas (misconceptions) recorded in their prediction and change their ideas as a result of new, convincing evidence. But research repeatedly demonstrates that there are other factors involved in rejection of an incorrect idea and that the process of conceptual change can be complicated by students' motivation, values, and beliefs. 19 Additionally, there are countless variations in how students may be assessed and many of these variations lie outside of this discussion.
We encourage formally testing these ideas, in particular at institutions where large-scale laboratory redevelopment is not possible due to constraints in time, money, and personnel. For example, how might verification labs impact learning if students do not yet know the concept they are attempting to verify and can meaningfully construct the knowledge (and predictions)? How do different assessment structures impact learning? Are there ways to reduce unnecessary cognitive load in labs to improve learning? What structural manipulations would improve interactivity? In the next section, we present some possible manipulations based on the discussion here.

B. Implications for instruction and research
From this analysis, we can suggest general implications for instruction, be it labs or demonstrations (Figure 2). While these suggestions have already been shown to be effective in the case of enhanced and Interactive lecture demonstrations, there are some limitations for improving verification labs.
Instructors may consider making minor changes to existing lab instructions that require students to generate their own physics predictions without grade penalties. However, scheduling labs so that students have not yet received instruction, but soon will, is complicated with large multi-section courses. Assessment should reward generation, positive interdependence in a group, and provide feedback, rather than correctness or attendance. Training lab instructors to facilitate interaction and to evaluate reasoning is non-trivial. Providing opportunities for revision is time intensive in a lab and may be further limited by the available equipment.
A key question concerns resources and efficiencies: is it worth it to modify labs and continue to use them to teach or reinforce content? The research suggests that learning could be more efficiently achieved through a 15-20 minute enhanced (or Interactive) lecture demonstration, rather than a two-to three-hour lab. Thinking back to the discussion of cognitive load, there are many other features of labs that do not exist in demonstrations or other forms of instruction and are valuable to instructors. 20 As argued in many other places, 21-23 labs may be more effectively used for focusing on skills that can be uniquely learned through the activity associated with labs: understanding data and measurement, working with equipment, designing experiments, and critically evaluating models and data. The American Association of Physics Teachers has compiled a set of recommendations for undergraduate physics labs that provide a starting point for instructors. 21 The Physics Teacher and the American Journal of Physics also have many articles with practical applications of those ideas such as those in Refs. [24][25][26][27][28] .
Research results demonstrating no measurable impact of verification labs for reinforcing physics content knowledge should motivate departments to critically evaluate the role of lab courses in undergraduate instruction. It is also important to be critical of such results, making sure we understand the mechanisms for outcomes. We hope the discussion here sheds some light on the potential aspects of verification labs that are limiting the learning students can achieve and provide some ideas for making valuable use of student time in labs.

ACKNOWLEDGMENTS
This work is supported by the Cornell University College of Arts and Sciences Active Learning Initiative. We would also like to thank the feedback and contributions of the Cornell Physics Education Research Lab, Dr. Carl Wieman, and Dr. Kate Follette.