{'fraction_non_alphanumeric': 0.024308258438893213, 'fraction_numerical': 0.006333592810133748, 'mean_word_length': 5.2848565710473645, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Best practices for supporting student experiential learning in an online environment during emergency situations: Lessons from Covid-19 pandemic response

Background: In 2020, nearly four weeks into the first semester of the academic year, the University of Canterbury (UC) in Christchurch, New Zealand, transitioned to online delivery of all teaching in response to the nation-wide lockdown due to Covid-19. With 48 hours’ notice, all on-campus activities ceased and were replaced with lectures delivered live via Zoom or recycled from previous delivery, and simulated, or otherwise facilitated, practical experiences. Assessments were temporarily halted, then resumed over a more compressed timetable with online and “take home” activities largely replacing traditional invigilated assessments. Purpose: Our research explored the challenges and successes of UC’s fast transition to online learning, with a particular emphasis on understanding the impact of the loss of face-to-face delivery on practical experiences normally delivered through laboratory and field activities. Our goal was to develop a set of best practices to improve our response to future emergency situations. Methods: A two-part survey was electronically distributed to students enrolled in a 4-year engineering programme and a 3-year sport coaching programme at the University of Canterbury. Results: While students strongly appreciated efforts of lecturers and tutors to support their learning, differences in communication approaches affected student experience both positively and negatively. A clear need for two-way communication as well as consistency and clarity in messaging were key findings. A general dissatisfaction with replacement practical activities was expressed, demonstrating a need for contingency measures to be in place to mitigate the impact of future disruptive events on practical learning. Efforts to manage online assessment of learning were generally well received, even in a compressed timeframe, though several unhelpful practices were highlighted. Students also identified many positive aspects of their learning experiences, including live tutorials and madefor-purpose video presentations.


Introduction
Our research explored how students were affected by changing from on-campus education amongst peers to a distance learning environment. In particular, we examined the impact on students in regards to communication methods, practical activities, and assessment. The aim was to understand what might be best practices in delivering education and assessment via online and distance learning methodologies in strongly practice-based courses.
Modern tertiary education typically includes strong elements of experiential learning such as laboratory and field-based experiments, work placements, and nontimetabled group projects in addition to traditional timetabled lectures and small-group tutorials, as well as online activities. The latter includes discussion forums, Wikis, breakout activity groups, and shared screen projects. Experiential learning supports different student learning styles, may elicit deeper learning and engagement, and enables students to participate in activities which they may undertake in their careers following university study (Gadola & Chindamo, 2019;Kolb, 2014;Petersen & Clarke, 2015). Much informal learning occurs in practical sessions through discussion among peers, as they collaboratively explore the practical environment (Wilson, Ho & Brookes, 2018). Like other tertiary education programmes in sport coaching and engineering, UC requires students to undertake internships -work placements in professional organisations which serve as an introductory apprenticeship in the workplace and often as an extended job interview (Heiskanenv, Thidell & Rodhe, 2016).
As with other tertiary settings, at UC summative assessment occurs through a variety of activities, including student presentations and reports, submissions from group and solo projects, traditional invigilated exams, and online quizzes. Other assessment activities include class discussions, presentations, and in-class quizzes (Meyer et al., 2010). The research that informs this paper took place in 2020 after a nationwide lockdown for Covid-19 interrupted the first semester and required an abrupt transition to online delivery of all teaching, labs, and practical experiences. Most lectures were either given via Zoom or recycled from prior years' recordings, but experiential learning and assessment were temporarily halted then resumed over a more compressed timetable with a large amount of online and "take home" activities replacing traditional in-person activities and invigilated assessments.
Although faculty and teaching staff met online frequently to develop an emergency plan to deliver the essential content for all courses, the students were not consulted, and thus we wanted to survey the students to better understand which activities, teaching styles, and assessments best facilitated student learning.
Our research sought to answer: 1) What were the key communication strategies employed by staff that positively impacted student experience? 2) What were student perceptions of the impact of the move to distance learning on practical work? 3) What were student experiences of assessment following the disruption and subsequent move to online assessment?

Methods
All undergraduate students across a 4-year Engineering programme and a 3-year Sport Coaching programme at the University of Canterbury (UC) were invited to participate in two surveys. This research was approved by the University of Canterbury Human Ethics Committee, and all participants provided informed consent prior to completing the surveys. UC divides the calendar-based academic year into two semesters and divides each semester into two distinct terms of six weeks each, separated by a two-week break. Semester 1 from mid-February to the end of May thus has Term 1 and Term 2, and Semester 2 from mid-July to the end of October has Term 3 and Term 4. The aim of the first survey (n=78 respondents), distributed shortly after the announcement of the New Zealand lockdown four weeks into Semester 1, was to gain information about students' experiences of predominantly on-campus delivery in Term 1 of 2020. The second survey (n=258 respondents) was opened immediately after Semester 1 finished, capturing responses to distance learning that occurred in Term 2.
In both surveys, the students were asked a combination of Likert-style questions about their perceptions of various technologies, communication methods, and assessment strategies used in their courses. Several open text questions were used to gather more detailed information about their experiences of learning in an unplanned distance environment.
Likert-style questions asked students to rate their agreement with statements regarding aspects of their learning experiences, including; communication methods, assessment, and practical activities. All such statements were framed in a positive manner, with seven available response options: Strongly Agree, Agree, Slightly Agree, Neither Agree nor Disagree, Slightly Disagree, Disagree, Strongly Disagree. Thematic analysis (Clarke & Braun, 2017) was used to analyse open text questions by reading through comments and then categorising into emergent themes.

Results
Strong emergent themes included 1. quantity and quality of communication from teaching staff, 2. loss of practical and workplace activities and inadequacy of replacement experiential learning opportunities, and 3. challenges of assessment via online exams.

Communication
Effectiveness of lecturers' communication was one theme that emerged from student responses regarding positive experiences during Term 2 distance learning. Students felt empowered when they had a clear understanding of assessment details, were told the timing of available lecture materials, had the ability to ask questions of teaching staff, and received timely feedback on assessments.
Students were asked "What were the primary ways in which your lecturer communicated course information to you during your distance learning in Term 2, and how effective were these?" Five prepopulated responses (Table 1) were provided as well as the opportunity to add further responses in a text "other" category. Participant responses were categorised from the 7-point Likert scale responses into three combined categories of "Positive" (Effective/Very Effective), "Neutral" (Slightly Effective/Neither Effective nor Ineffective), and "Negative" (Ineffective/Very Ineffective). In Tables 1, 2, and 3 colour has been added to aid reading. Positive responses are indicated in blue, neutral in green, and negative in red. Darker colours indicate higher percentages of students gave said response. Based on comments from students, it was apparent that uploaded materials and prerecorded lectures were considered most effective, as they allowed for asynchronous learning outside of traditional university hours. The students appreciated being able to watch multiple lectures in a single sitting, pause a video as needed, and have access to all of the materials on line.
The most disliked form of communication was use of pre-recorded lectures from prior years. These caused issues, as the material was not fit for purpose for the 2020 cohort of students and didn't take into account the distance learning environment. Live Zoom lectures, although well-received by about a quarter of the students, posed challenges for others. This appeared to be tied to timetabling and the inability of students to keep up with content as it was delivered live, although it was recorded and available for access later. Thus, in lieu of attending live, many students used the recordings to watch the lectures.
Comments from students regarding communication varied. Most of the positive comments lauded clear communication from their lecturers, as in this typical student response: Some lecturers provided clear communication on the changes, posted material early in easy to find (consistent) locations and provided a mixture of live and recorded material which was done specifically for this year's course so it was all relevant. (Student 15) Meanwhile negative responses tended to reflect a perceived lack of communication, with one student reflecting on different communication strategies employed between the teaching staff of two departments: There was a lot of variation in the quality of distance learning. … As a student in two schools … [Department name] lecturers were constantly communicating, providing guidance, making sure everybody was aware of everything that needed to be completed each week, checking in etc. Material was easy to find, easy to follow and there was plenty of Zoom sessions available to ask questions, follow up etc. One [other department] lecturer provided a good amount of communication and a nice mix of live and pre-recorded material. The other was diabolical and basically just posted all the old lecture material up (that was poorly recorded) and left us to it. It felt like they had pretty much stopped their teaching responsibilities. When they did run a tutorial they failed to make it clear how to join … and I missed the opportunity. (Student 13)

Experiential learning
Students tend to choose UC for the opportunity to undertake practical activities and workplace internships, and it is a major factor in their decision-making for course of study. Of the 77 responses to the statement "The prospect of undertaking practical activities strongly influenced my decision to study at the University of Canterbury," 95% were neutral or positive, with 69% agreeing or strongly agreeing.
The strong emphasis placed on experiential learning in Engineering and Sport Coaching programmes at UC was reflected in student responses for their Term 1 experience (Figure 1, left). When asked, "What were the most effective parts of your oncampus learning experience?" -a question which had 13 prepopulated options (Table 2) along with an 'Other' field -71% of students identified 'Laboratory/Practical sessions' as most effective, with an equal proportion (71%) identifying the importance of 'Attending lectures.' The only other option identified by more than half the total survey respondents was 'Social Interaction with other students,' identified by 65% of students. Table 2 Agreement with "What were the most effective parts of your on-campus learning experience?" In the second survey, distance delivery of planned practical activities in Engineering and Sport Coaching included experiments, analysis of lecturer-provided data, or viewing videos which demonstrated what would have been covered in class. These were, unsurprisingly, less well-received by students (Figure 1, right) with a significant number (42%) reflecting quite negatively on their replacement experiences.
Students were asked to identify which laboratory/practical activities they participated in during the Term 2 distance learning phase of Semester 1. Table 3 shows the responses to the two-part question. If the activity was not used, the students selected "N/A." If the activity was used, then they were asked to rate it. Six options were provided: 1) At-home physical experiments, 2) Use of phone and/or computer apps, 3) Recordings from previous years, 4) Specially-recorded lab videos, 5) Lecturer-provided data, and 6) YouTube videos of related activities. All options had a high use rate with over 100 responses, and only a handful of students identified other practices beyond the six listed. For all options, approximately half of the student responses were in the range from Slightly-Satisfied to Slightly-Unsatisfied.  Figure 1. Student experiences of practical activities on-campus for Term 1 and first survey (left) and by distance for Term 2 and second survey (right) in Semester 1 2020 The most widely used replacement for data-taking experiments was for the lecturer to provide prepared data sets for the students to analyse (166 responses). These data sets came from a variety of sources, including from data sets recycled from students who took the course in prior years or who had completed the lab prior to the lockdown. This was well-received with almost 85% positive (108) or neutral responses (13). Only 15% of respondents (45) reflecting negatively on analysing pre-prepared data that was not their own (Table 3). Another popular option was the use of specially-prepared videos of practical sessions (125 responses). This had almost identical responses, with 85% positive or neutral about the prepared videos. Amongst students, the most popular practice for replacement labs or practical activities was to provide YouTube videos of related activities from other sources (124 responses, 40% positive, 46% neutral). On the other hand, students struggled to engage with conducting at-home experiments (120 responses, 13% positive, 33% negative). The least used option by lecturers in their courses was using videos of students participating in the lab in prior years (106 responses); however, when it was used, there was a mixed response (69% positive or neutral, 31% negative). This seems at odds with the rest of the responses, so we assume the videos from prior years must have been of poor quality, were mostly lecture and/or demonstration, or did not adapt well to an online learning environment. Table 3 Student responses to survey question: "Please describe the laboratory/practical activities you participated in as part of your distance learning coursework in Term 2, and please rate how effective you found these?"

Assessment practices
When the nationwide lockdown occurred, the university closed and UC lost nine days of teaching prior to the normal two-week mid-semester break. Most Engineering and Sport Coaching courses use the last several days of Term 1 for a first major assessment of student learning, either as a test or a large report, so a substantial number of assessments were abruptly postponed until "later," leaving students confused as to what was expected of them during the mid-semester break.
Only five of the lost nine days were recovered by resuming Term 2 a week earlier than scheduled. This was compounded by the university campus remaining closed and all classes, labs, and practical activities being offered in an unfamiliar online format that was hastily put together by faculty also unfamiliar with teaching online. UC recognised the loss of teaching time and acknowledged the many challenges students faced during the lockdown and implemented an institutional policy of deferring all missed and pending assessment dates until much later in Semester 1. This led to a significant assessment "crunch" towards the end of the semester where students, now trying to manage their studies in an online environment, ended up having multiple high-value assessment items delayed and thus due in a very short space of time with little coordination among teaching staff to stagger the new due dates. Some students reported feeling overwhelmed and that they lacked strategies to regulate and structure their learning in this new environment.
Assessment via a written test also proved challenging in an online environment. Given the short time frame and cost involved, implementing proctoring services was not an option, and course lecturers adopted a range of approaches, with mixed responses from students. Table 4 shows student responses to a question regarding assessment task types, and their satisfaction with those activities. Students were prompted for a rating only if the assessment task type was used in their classes. It is heartening that the most used activities (open-book timed test and written assignments) were the most popular. Student discomfort with presenting work live online, including in the relatively safer environment of a group, generated the most negative feedback. This aligns with anecdotal feedback from colleagues during the lockdown period who reported students being unwilling to have their video or audio feeds activated during Zoom sessions. When attending class in person, students all face towards the lecturer and no one is looking at them. In a Zoom session, students felt exposed, as everyone could see and hear them, their living environment. Coupled with numerous embarrassing viral videos of people forgetting to turn off their cameras or sound at crucial times, and it was not surprising that most students made themselves invisible and muted during class lectures. The strategy of sequential exams (where questions are asked in sequence with no ability to go backward or forward) proved particularly stressful to students, as reflected in the following survey response: The first test [we took] in lockdown ... was sequential (so we could not go back to check or change any answers) and negative marks were given for incorrect answers. ... We were already feeling stressed and alone, and measures supposedly meant to prevent collusion just fed that anxiety. (Student 137) Lecturers providing a tight timeline for test completion to avoid cheating by students was also poorly received. Students cited technology difficulties, slow internet speeds, and inadequate time to upload exams as contributing to their overall stress and anxiety. Reweighting pieces of assessment to replace lost assignments and activities also arose as an issue with students. There was a perceived lack of fairness about making a smaller number of assignments worth an increased percentage of their grade, especially assessments completed before the lockdown.

Communication
In a distance learning environment, especially in an emergency situation where anxiety levels are already high, it is important to have clear, frequent communication with students. This is best provided in an online location students can access asynchronously.
It is also important to provide timely feedback in an accessible location so that students can monitor their own progress. Finally, it helps for the lecturer to provide live drop-in question and answer periods for students to assist them in their learning.
Some key communication best practices were to accommodate for the loss of office hours, peer access, group study, and hands-on learning experiences. One Engineering lecturer was highly praised for his use of specially-prepared video lectures that he posted online in weekly blocks well in advance. The lecturer then used the regular class period as a video-recorded Q & A session where students could ask him to explain concepts, work through examples, or provide guidance on homework assignments. Students could passively participate in the Q & A sessions and still benefit from attendance, or they could actively ask questions, conduct side discussions with peers in the chat window, and otherwise recreate the feeling of group study. The pre-recorded lectures also allowed students to watch all of the week's lectures in a block, pausing and backtracking through them as needed to absorb all the content.
Another communication strategy that worked quite well was a blog. Either through the course webpage or another online location, interactive Q & A and student-driven response options were popular as an aid to learning. Especially with experiential learning activities, having access to other's solutions to analysing data proved helpful.

Experiential learning
Practical activities provide a point of difference for in-person learning universities and are evidently a strong determinant of student enrolment and satisfaction. The period of distance course delivery in our study was due to an exceptional unpredictable event, and many student comments did commend staff for their efforts in providing replacement experiences. Nonetheless, students suffered from the abrupt transition and the university's lack of preparedness. Students' low satisfaction with replacement practical activities is a concern for retention of students, particularly in the first-year cohort, as these students have often not established strong personal ties to the university, their peers, or their lecturers. Findings arising from this study suggest that research into and preparation/planning for potential future disruption from events forcing study offcampus, should strongly focus on provision for satisfying, engaging practical experiences.
Electronic interactions or offline virtual experiences appear to be the most engaging options for student experiential learning. Having these available is prudent even when inperson learning is available, as it provides options for students who miss a lab or have a disability that prevents their participation.
Preparing for a future disaster or emergency event that forces education off campus would also have pay-back in the present, in that it would enable the institution's existing programmes to be effectively delivered remotely to a distance cohort, thereby increasing the potential reach of the programme. In the Sport Coaching programme, which already has a distance cohort, it was observed that these students felt their learning was enhanced as they felt more included and engaged with the rest of the class.

Assessment
Institutions need to consider how to adapt face-to-face exams to an online format. With online versions of an exam there are different factors at play: technical difficulties, delivery and transmission of materials, and time required to think through and answer a question in a different format. Academic honesty and integrity are a challenge, as institutions want to ensure they are graduating students who have met accreditation standards, especially in fields like engineering where an unqualified person could design something that puts people at risk. It is also important that assessments are not putting students at an unfair advantage or disadvantage based on their socio-economic status, access to peers and study groups, access to technology, or familiarity with technology or software.
In today's world, students have many tools at their disposal that facilitate cheating. As examples, textbooks are available as PDF copies and can be scanned for keywords to answer any fill-in-the-blank question in seconds; computers, laptops, phones, and friends can provide ready answers on the fly; screen captures of questions and answers can be shared on the web or with friends; old tests, labs, reports, and files can be stored and given to peers as resources; and students can provide their login details and pay others to do their online exams for them.
The use of sequential exams, although a disaster, was implemented by some lecturers at UC as a method to discourage cheating. By making students answer questions in series with no option to go back or skip ahead, and by enforcing time limits, it was easier to ensure that the student was doing their own work. Other strategies that were tried but did not work well were web cameras that kept track of eye movement (to ensure the student was looking only at the screen), face invigilation (where the student's face was visible), and time limits for each question or the overall exam. All of these methods created a lot of stress, and when technical difficulties occurred, such as the entire computer system slowing down due to everyone trying to upload their answers in time, it was evident the entire assessment methodology needed to be revamped.
In addition, students found assessments in the form of live presentations and oral exams confronting, particularly in an online session when they couldn't see the audience, and when they felt exposed to their presentations potentially being recorded (even if the lecturer was not recording them). As presentations and oral exams are a good option for experiential learning activities, lecturers should make efforts to increase student comfort by using individual sessions or small breakout groups, and they should accommodate students with disabilities who are disadvantaged by a live format, such as by allowing them to provide a written document or a pre-recorded presentation to demonstrate their full knowledge.
Although no one likes tests as a form of assessment, they have numerous advantages over other types of assessment as they can ensure the student has individually learned the material of the course. Unlike research reports and projects, a test makes it clear the student understands key concepts and can demonstrate their knowledge without any outside help, such as proofreading, editing, or text "lifted" from another source. Tests are also quicker to grade, which is critical for large classes.
Thus, universities need to invest in ways to improve the use of invigilated assessment and tests in online platforms. UC is investigating the use of software that locks down the student's browser, preventing them from accessing the internet, although this does not prevent them from using their phone or another laptop, so the lockdown would be enhanced with face invigilation to check the student in view is in fact the student registered to take the exam. UC is also tracking IP addresses to make sure the IP address is local.
Another idea that shows promise is online exam "banks" that contain hundreds of potential questions. Each student would get their own custom exam that was statistically proven to be equivalent to everyone else's exam in terms of level of difficulty and inclusiveness of covered content. Yet another idea that is promising is an exam that has randomised sequencing and also uses different values for questions requiring calculation. Nothing is fool-proof, but making efforts to eliminate cheating without adding stress to students is important to attempt.

Concluding comment -A suggestion
Our research showed that the university generally handled the disruption to normal inperson delivery well, however, it also revealed a number of practices we could implement that would be beneficial in a future lockdown or emergency situation. Our research suggests tertiary institutions should ensure their contingency planning includes preparedness to implement these best practice behaviours and activities to better support student learning in such challenging situations.