{'fraction_non_alphanumeric': 0.02168992832893248, 'fraction_numerical': 0.0038193134666163714, 'mean_word_length': 4.79006279006279, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Flipping Freshman Mathematics

Abstract Our study compared a flipped class with a standard lecture class in four introductory courses: finite mathematics, precalculus, business calculus, and calculus 1. The flipped sections watched video lectures outside of class and spent time in class actively working on problems. The traditional sections had lectures in class and did homework outside of class. No statistical difference was found in test scores of the students, though qualitative data indicated potential problems with implementing the flipped pedagogy. Specifically, many students had negative opinions of the flipped model, and attitudes toward math in general tended to decline, comparatively, for students in the flipped class.


INTRODUCTION
Much of the research on implementing flipped classrooms has demonstrated benefits for student learning. For mathematics courses in particular, flipping has been shown to be effective in linear algebra [3], calculus 2 [5], and mathematics for elementary teachers [2] courses. In those studies, one section of the course was taught in a traditional manner and the other section was flipped. The research indicated students in the flipped section performed equally well or better than students in the nonflipped class, and students in the flipped section had positive feelings toward the flipped method of instruction. Other, more qualitative research has shown that students in a flipped statistics class became more open to cooperative learning and innovative teaching methods, but they were less satisfied with how the flipped class oriented them to the learning objectives of the course [6].
During the Fall of 2013, four mathematics courses were flipped by utilizing the screencast software Camtasia to record videos of lectures. Students watched the videos to learn the material before coming to class, and spent time in class working on homework problems. The flipped class has many potential benefits for students, but since the concept is still fairly new, relatively few studies have investigated the impact of the flipped methodology on student learning. Our goal is to add to that body of research by comparing flipped and traditional sections of four different core math courses.
Although there has been some research on flipping mathematics courses, much of it has been in higher-level courses. Strayer [6] warned that an inverted classroom may not be the preferred design for an introductory or general education course. One of the motivations of our study was to test that hypothesis by flipping introductory math courses. Just as in studies done in [2], [3], [5], and [6], each professor taught two sections of their course, one using a flipped method of instruction and the other taught in a traditional lecture style. Our research did not demonstrate all of the benefits shown by the other researchers. On the contrary, our quantitative research demonstrated no difference in learning between the flipped and nonflipped sections. The qualitative research also showed that our flipped implementation engendered negative feelings in many of the students and generally lowered their opinions of math.

METHODS
Our institution is a medium size, private university with a focus on undergraduate education. Each of the four instructors involved in the study taught two sections of a freshman-level, general education mathematics course. The four courses were finite mathematics, precalculus, business calculus, and calculus 1. Each instructor taught one section of the course using his/her traditional style and the other using a flipped classroom approach. The students in each section took equivalent exams and completed largely the same homework assignments.
The main tool for content delivery in all the nonflipped sections was via lecture, although aspects such as frequency of in-class quizzes, homework discussion, and group work varied by instructor. Students in these sections learned all new material while in class with the instructor. They were then given homework assignments to complete outside of class.
The instructors prepared lecture videos for the flipped class using screencast software. Computers, software, and microphones were provided by an internal grant from our university, which guaranteed that each of the flipped courses was taught in a relatively similar manner. The content in the videos was identical to the content covered in the lecture-based section of the course. Students in the flipped section of each course were asked to watch videos on a particular topic to prepare for class. Instructors encouraged students to take notes while watching the videos, though this was not typically a graded requirement.
In an attempt to ensure that students watched the assigned videos, instructors gave a short assignment due before class. These were administered using the WebAssign online homework system. These "pre-assignment" problems were intended to be easy to medium-level problems that allowed students to check their basic understanding of the concepts in the videos. Students received no credit for late pre-assignments. The final question in the pre-assignments was open-ended, allowing students to express what they found confusing or interesting about the lesson.
In each of the four flipped sections, instructors made individual choices about how to use class time. Instructors often began class by addressing questions that arose via the pre-assignments. This was designed to clear up any confusion students had about the videos and/or pre-assignment questions. Some instructors gave regular quizzes, whereas some designed activities centered around the day's topic. Some instructors used the majority of the class period for students to work on their online homework assignments. If students finished work early, they might be assigned a challenging problem or they might go on to the next set of videos. Some instructors utilized groups in a formal way, whereas others did not. The common thread among the four flipped sections, despite the differences in course content and instructor style, was that students were actively working on problems during the entire class period. Whether online or on paper, in formal or ad hoc groups, students were actively engaged during the class hour.
Students in the flipped sections were expected to finish homework assignments on the current topic within a couple of days of class, although the timeline for these deadlines varied by instructor. The cycle then began again with a new set of videos before the next class period.
Near the start of the semester (around week 2), each instructor invited his/her students to participate in our research study. We administered two surveys to the students who chose to participate. The first survey was the 40-question Attitudes Toward Mathematics Inventory (ATMI) [4]. This survey was given once at the start of the semester (approximately week 3) and again at the end of the semester. We then designed a survey aimed at gaining information about student habits and attitudes toward various teaching styles. This survey asked students about their study habits, preparation for class, and opinions about class structure. This survey was given once in the middle of the semester and again at the end of the semester.

Quantitative
At the end of the semester, a two-way ANOVA test was performed on final course grade, final exam grade, and test average. No statistically significant difference was found in the data between the traditional and flipped sections of each course.

Qualitative
In order to analyze student attitudes towards mathematics, we selected 16 statements of interest from the ATMI survey and analyzed them. See Appendix A for details. Of those, the results that were statistically significant all point in the direction that the general attitudes in the flipped courses were more negative than those in the traditional lecture-based classes. In some cases, the nonflipped classes showed an actual improvement in attitudes and the flipped courses showed a decline.
We surveyed students using both the ATMI survey and a survey we designed to assess student habits and opinions. Although we expected opinions would be mixed, the results were rather striking when we directly asked students if they preferred the flipped method. In the mid-semester survey, roughly 45% of students in flipped classes reported a preference for the flipped class to a more traditional class. However, at the end of the semester, only 36% reported that same preference. This indicates a shift away from a preference for the flipped class as the semester wore on, which could be evidence of student fatigue in regards to the new method of instruction. See Appendix B for details.
Data also indicates an interesting difference among the four courses, as finite mathematics had results that diverged from the other three courses. Comparing the flipped and traditional sections of finite mathematics, a greater percentage of flipped students felt their interaction with the professor was helpful and that the class atmosphere was effective for learning. In the other three courses, though, the opposite was true, and the discrepancy was especially pronounced in the precalculus course. See Appendix B for details.

Free Response
The surveys also included some free-response questions in an effort to encourage students to voice their opinions about both the videos and the course format in general.
When asked what was most beneficial about the videos, many students highlighted that they could easily stop, rewind, and/or pause the videos while watching them and taking notes. Students also frequently mentioned the benefit of being able to go back later to reference the videos when studying for an exam. When asked instead what changes could be made to improve the videos, the most common response was to say that no changes should be made.
More generally, students were asked to comment on the benefits of the teaching methods used in the class. Students in the flipped sections again mentioned the benefits of the videos themselves, but also frequently referenced the ability to work on problems and to ask questions in class. We also asked students to comment on the drawbacks of the teaching methods used in the class. The most frequent response from students in the flipped sections was that they could not ask questions directly to the instructor while taking notes from the videos. Example comments appear in Appendix C.

LESSONS LEARNED
Although we did not share all of the successes with inverting our classrooms that several studies before ours did, we feel as though there is valuable information and experience to be gained through our study.
1. We concur with Strayer [6] that freshmen may not be the best population for implementing a flipped class, especially first-semester freshmen. Approximately 81% of the students in the classes we flipped were firstsemester freshmen, and the classes most hostile to the flipped methodology coincided with those that had the most freshmen. In particular, Tables 4 and 5 (see Appendix B) indicate that the largest discrepancy between the flipped and nonflipped sections occurred in the Precalculus course. The flipped Precalculus section was the only one that had 100% freshmen, which may partially explain why that section had such a low opinion of the class atmosphere and interaction with the professor. Since freshmen have so many other adjustments to make, they may not respond well to a dramatically different teaching style. 2. All students did approximately the same amount of work outside of class.
Despite some students' perceptions in the flipped classes that they were doing more work, self-reported data did not validate their claim. 3. Stress to the students that the teacher is actually teaching through the video, students do not have to "teach themselves" the course, and that this is not an online course. Several students commented that they felt like they did not have a teacher for the class, or that the teacher did not actually teach. 4. Make your own videos. In other papers it has been mentioned that professors can get their videos from other sources, such as You Tube or Khan Academy [1], [5]. In our surveys, students indicated a strong preference that their own teacher make the videos. This made it easier for the students to seamlessly transition from the videos to in-class activities. Although we may feel that we all teach the same material, students often perceive differences in what each teacher emphasizes. This could explain their desire to get the material straight from the source, and may create frustration for students watching videos not made by their own instructor. This conclusion corroborates other research that the out-of-class and in-class portions of a course must be well integrated [6].

5.
Have a policy in place to ensure that students watch the videos. This can be accomplished with an in-class quiz at the beginning of the class period and/or with regular note checks. In our study, only one instructor checked student notes for a grade. 6. Reduce randomization for problems worked in groups. Much of our in-class activity included working problems from an online homework system. These exercises frequently had randomized numbers and this sometimes caused difficulties with students working in a small group. They did not always feel they were working on the same question since they were not headed towards the same answer. 7. Design a wide variety of in-class activities to ensure that students remain engaged through the whole class. Thus, if students finish the in-class problem set early, they can transition to another activity. Also, if students are simply working problems the whole time, it is not likely they will remain engaged for the entire class period. 8. Class time must be set up so students who did not watch the videos are still able to benefit from attending. Although there should be consequences (points lost on the quiz, etc.), students should not feel like they cannot participate on a given day if they did not watch the video. 9. Do not treat the method as anything new to you. In order to use student data for our study, students had to sign consent forms. Therefore, there was no way for them to not know that this was as new a method for us as it was for them. Our students did not like feeling like they were an experiment. 10. Remind students of strategies for effectively using the videos. Since this is a new way of learning for most of the students, they may need to be introduced to these studying techniques on the first day of class, and then reminded a few weeks later. For example, it may help to mention again that taking notes during the video is beneficial, or to point out a problem on the test and how it connected to a particular video. 11. The instructor's view of interaction is different from the student's.
Although we felt we had much more interaction with students in the flipped classroom, the results in Table 5 (see Appendix B) demonstrate that students in the flipped class did not always share that feeling, as three out of the four courses had lower ratings for interaction in the flipped section. One other important element involved in the results from Table 5 is the presence of supplemental student instructors. Each of the flipped sections, except for Precalculus, had a student assistant who roamed the class to answer questions along with the professor. This may help to explain why the interaction scores in the flipped Precalculus section were so low, because students were getting on average half as much assistance during class. These results also illustrate that teaching assistants may have a larger impact on attitudes in a flipped class as opposed to a nonflipped class. 12. Students may resist becoming more self-sufficient learners through the flipped class. One of the potential benefits we perceived of the flipped model is that it can create more self-sufficient learners by exposing students to new material outside the classroom and encouraging students to come to class prepared. Our anecdotal evidence indicates that this potential benefit did not materialize for most of our students. In fact, some students were actively resistant to taking ownership of their learning.
In the semester following our trial, our lessons learned were used to improve our flipped classrooms. One professor removed the electronic aspect and had the students work problems on paper during class. Another used the screencasts to introduce the material for new sections, presented a mini-lecture on the subject in class, and then had class activities. Although no data was gathered to support any conclusions regarding these classes, both professors reported that the class ran more smoothly, and students had more positive attitudes than in the previous semester.

CONCLUSIONS
There were many lessons to be learned throughout the semester and we all gained a wide variety of ideas that could be adjusted and implemented in future classes. There seems to be a movement in higher education towards the flipped methodology, and we think there is a place for some aspects of this model in mathematics as well. Our research indicates that flipping the classroom for the entire semester might not be successful for freshman-level courses. However, there are still topics in these courses that can be presented with this technique, allowing more class time to be used for active learning. We feel that the best method in freshman-level math courses might be more of a blended approach. This would allow some responsibility to be pushed towards the students, providing an easier transition to collegiate-level courses, while at the same time alleviating disengagement associated with the day-to-day monotony of a typical class.

APPENDICES APPENDIX A: ATMI SURVEY ANALYSIS
The ATMI Survey is a 40 question, four-factor survey designed to measure attitudes toward mathematics. We selected 16 questions from the survey that we felt pertained the most to our study. A pre-post analysis was performed to see if there were any changes in attitudes in individual students. Next, a comparison between the end-of-semester responses in the flipped and nonflipped sections was performed. A two-way ANOVA test was used in both cases. Tables 1 and 2 show the statements that resulted in a significant p-value. If the statement also had varied results across the four different courses, their interaction p-value is listed. In addition, the last column reports the mean difference in the prepost analysis and the mean value in the post analysis on a scale from 1-5 where 1 = Strongly Disagree, 2 = Disagree, 3 = Neutral, 4 = Agree, and 5 = Strongly Agree. In the final column, f = flipped class and n = nonflipped class.
Below is a list of statements from the ATMI survey for which data showed no statistical significance between the flipped and nonflipped classes: • I get a great deal of satisfaction out of solving a mathematics problem. • Mathematics helps develop the mind and teaches a person to think. • Mathematics is important in everyday life. • Mathematics makes me feel uncomfortable. • I have a lot of self-confidence when it comes to mathematics. • I am confident that I could learn advanced mathematics. • I like to solve new problems in mathematics. • I am comfortable expressing my own ideas on how to look for solutions to a difficult problem in math. • I am comfortable answering questions in math class.

APPENDIX B: FURTHER SURVEY RESULTS
At the end of the semester, 113 students completed both the ATMI and the survey we designed to assess student habits and opinions. Of these students, 64 were from the traditional classes and 49 from the flipped classes. A few items worth noting are displayed in Tables 3, 4, and 5. Table 3 reports students' preference for a flipped versus traditional classroom at the middle and end of the semester.  Table 4 reports the percentage of students responding either effective or very effective to question "How Effective was the General Atmosphere of the Class to your Learning Experience?" Lastly, as instructors, we felt we had more interaction with our students in the flipped classroom, however, results indicated students in the flipped course felt less engagement with the instructor, as shown in Table 5.

APPENDIX C: EXAMPLE FREE RESPONSE COMMENTS
The comments below highlight the most common types of responses we received from students' free response answers on our survey.
• If there is one concept in particular that you do not understand, you can pause and replay that section of the lesson as many times as you need. (As opposed to in class, where you can't always ask the teacher to stop and 'rewind.') • [The videos] helped because the student has access to the knowledge whenever they want, so it makes for a great way to review for an exam. • You have more time in class to do math problems. • [We] used the in class time to ask questions about the more complex parts of the material. • The video lectures make it impossible for people who are confused to ask questions and get an immediate response.