{'fraction_non_alphanumeric': 0.03140020663941422, 'fraction_numerical': 0.010421813934683976, 'mean_word_length': 4.961971076593465, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 4, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Flipped Learning in Information Technology Courses: Effectiveness and Impact

This study’s purpose is to investigate the implication of the use of flipped learning on students’ performance and perception. Participants are the second-year and third-year undergraduate students enrolled in the Information Technology courses. One hundred and twenty-five students were the participants associate with an experimental group and a control group. The experimental group consisted of the students of the ISD class A and BI courses. Meanwhile, the students of ISD class D and MIT were the control group. The experiments employed two instruments: (1) students’ final score, and (2) questionnaire. This study used the independent sample t-test and two-proportion z-test to analyze the collected data. The result showed that the students’ performance in the FL classes was superior to a traditional learning setting. Students’ perception of both courses is higher than the experimental group. Besides, we observed that studying in an FL model had a positive impact on students’ motivation. These findings suggest that the FL could be a promising way of enhancing students’ learning outcomes.


Introduction
Flipped learning (FL) is one of the innovative learning models that have been developed by Bergmann and Sams in the early-mid 2000s. This approach refers to a learning model that reorders how students spent their time in and out of the classroom by bringing lectures outside the classroom to provide valuable learning time [1]. The traditional way of teaching making students listening to lectures in class while having them reading textbooks and working on assignments outside of the classroom. In an FL, presentations, recorded lectures,and online readingsare assigned to the students by the teachers. This approach allows the students to assume the ownership of learning by learning on their own time [2].
In the last two decades, the availability of the internet and computers in the learning process has strengthened their commitment to using the computer to improve the learning outcomes. This strategy is believed to improve the student's cognitive abilities during the class. Most of the early studies in the FL evaluated the implementation in non-technological subjects, such as foreign languages, algebra, mathematics, chemistry, and others [3]- [7]. It happened because the researcher believed that the teaching material is more theoretical than technological courses. The concept of material is easier to understand, provided by the availability of online material [7]. It provides positive results for nontechnological subjects. Several studies on the effectiveness of FL also conducted on computer science subjects, including the introduction of programming (JAVA, XML, and advanced JAVA), software engineering, and computing I [8]. The ability to write and understand programs is an essential skill needed. The results showed that FL gave positive results in terms of students' performances and attitudes. The FL design also allows students to become more involved in the learning process. The 2 students also realized that they could understand the content independently at home before the class began [9].
At the beginning of 2018, our department applies FL as a learning model for some subjects. This study provides a review of the FL implementation. Besides, this study examined whether the performance and perception of students in the learning process varied from the traditional model previously used by the department.Lastly, this research explores the perspective of students to regenerate the FL model, which improves the FL model's teaching and learning efficiency.The results of this study could help improve the potential implementation of the FL model. This study addressed two specific research questions: (1) Does FL affects students' performance?; (2) how effective are the learning method of FL and its effect on the level of students' perception?
The structure of this article consists of: background on the introduction section and review of relevant literature on related works section, followed with a description of research methodology, result, discussion, and conclusion.

Related Works
A flipped learning (FL) is a part of a blended learning environment, in which students learn instructional content, watch video lectures at home and do the homework in class. At the same time, the instructors also provide customized instruction and collaboration with other students rather than lecturing [10]. Bishop and Verleger (2013) explain flipped learning as one that consists of two parts: interactive group learning activities within the classroom and an individual direct computer-based instruction outside the classroom. First, FL is an interactive learning group activity that goes into the classroom. Second, the outside class session is directing by an individual computer-based instruction. By allowing this approach, the students become more involved and responsible for their learning process. The students also get immediate feedback from the teachers during class activities.
The FL model, which creates a more learner-centered environment, indicated an impressive learning achievement. Moreover, it reflected behavioral improvement and enhanced communication in the FL classroom between teachers and students [10]- [11]. The FL enables students to learn course content with online educational materials such as video, film, and voices outside of the classroom environment. Also, this model allocates time for active learning activities such as question-answer, discussion, and problem-solving. Students are responsible for their progress and success in learning in the FL model [7]. Using this learning model, they increase their knowledge, enhance their comprehension, and increase their responsibilities for each course content [2], [4], [12], [13].
Research shows two primary advantages of implementing the FL. First, the FL replaces lecture time in class for active, collaborative, and problem-based learning and helps to improve the concepts of learning without reducing the content [14], [15]. Second, the FL can motivate students to become selflearners and life-long learners [7], [16], [17]. The researchers agree that FL has a positive influence on students. Not only improve students learning outcomes but also students' attitude and participation.
In terms of computer science courses especially programming courses, the findings study found that the FL model has a great potential to improve students' programming capability in introductory programming as well as students' academic performance [10], [13], [16]. The study showed that the FL model is suitable for an introductory programming course, where students find programming difficult [16]. It is beneficial using the FL classroom supported by technology. The approach provides the student with comprehensive access to valuable resources and learning material [13] in a medium that they believe applicable and appropriate to their learning style.
However, it also has several issues in the implementation. For example, students often find it difficult to complete online assignments and set their priorities in self-study sessions. Another one is most studies used video-based learning as a media of FL [3], [18]- [20]. Several study results it is known that not all of the students like the FL model that uses video lectures from the instructor (video record) as a media for delivering the material [18]. According to the students, the video only contains instructors who explain the material verbally and monotonously without any additional animation or visualization, so students feel bored, and the learning process becomes unattractive.

Hypotheses
Based on the above research questions, the authors formulated the following research hypotheses. : The experimental group will result in significantly higher students' learning performance when compared to the control group. : The experimental group will result in more extensive changes in levels of students' perception when compared to the control group

Participants
The participants of this study involved 98 students from two different Information System Development (ISD) classes, 50 students from the ISD class A as the experimental class, and 48 students from ISD class D as the control class. There also 27 students from the Business Intelligence (BI) class as the experimental group and Management of Information Technology (MIT) as the control group. In total, there are 125 students, and all of them have completed the survey questionnaires.

Course design and procedures
All of the courses (ISD, BI, and MIT) were six credits courses which taught twice a week. There are five ISD classes, but for this study, we only examined two classes, class A as the experimental group (used FL) and class D as a control group. Both have the same content material but the different instructors. Besides, we investigated BI class as an experimental group that applied the FL approach, and the MIT course became the control group. Students of the FL class learned the material by watching educational video lectures and reading other materials posted by Google Classroom. Later in the classroom session, students took part in face-to-face classroom activities such as group discussions and student presentations. The students were given several quiz questions as pre-class materials. In designing the online quiz questions, the instructor used the Google Form and Kahoot!. All classes have to complete five achievements.
In the FL model, students were assigned to watch or read the relevant lecture online before class. These lectures averaged 45 minutes in length that divided into 4 to 5 difference videos. It consisted of a PowerPoint presentation narrated by a lecturer. Students were encouraged to take notes and write down any questions they had. For viewing the entire term, all videos and materials are available online.
During the classroom session, the class began with approximately 10 to 15 minutes of quiz and 10 minutes of question and answer. For the next minutes, the students worked from the assignment on a problem that set in pairs or groups so that they can discuss it. While the pairs or groups worked on their problem sets, the lecturer, availed to students who had any problems or questions. This work was not collected offline, but the students are required to upload it online on Google Classroom.

Students' performance.
To measure the students' performances, we compare the final score of students. The final score is the accumulation of students' learning outcomes in exams, quizzes, assignments, mid and final test.

Students' perception.
At the end of the study, students were given a survey to measure students' perceptions ( Table 1). The students' perception in this study included students' background information about the participants, such as subject, class level, and their opinion about the learning method that used. Next, students were asked to rate their abilities to complete a particular task successfully and their level of understanding on the course content using a 4-point Likert scale (Strongly Disagree = 1, Disagree = 2, Agree = 3, Strongly Agree = 4). The students also were asked the learning media used in class. Then, the students choose the learning styles that work on them. At the end of the survey, the open-ended questions asked the students to give their viewson the learning process and their suggestions on how the Questions Students' ability to complete the task I was excited to study the course content before the class begins. I pay attention to the learning content during the learning process. I ask the teacher when having difficulties. I only study the course content material in-class sessions (face-to-face). I prefer to be silent when I do not understand the course content material. I only study for quizzes and exams. I enjoyed learning using discussion. I become silent when discussion sessions in groups. I want to learn to explore knowledge. I try to find references from other sources when I have difficulty in understanding the material. I enjoy exploring the material provided by teachers. Students' capability to understand the course content I know the course content that will be taught by teachers. I understand the relevant examples given by lecturers from the concepts taught. I understand the current issues in the field being taught. I can apply the subject matter in daily life. I can analyze the course content. I can apply the course content more broadly. I can do the quizzes and test that given well.

Data analysis
The Statistical Package for Social Sciences (SPSS) has been used to analyze the data. The students' performance was summarized using descriptive analysis by present the mean ( ) and standard deviation ( ). Independent samples t-test was applied to examine the students' performance both in the experimental group and control group. We put forward a significance level of α= < 0.05.
Responses to the closed-ended questions in the survey were analyzed using a two-proportion z-test. The test was used to determine if students' perception of their ability to complete a specific task differed across experimental and control groups. Two proportion z-test test was also used to determine if students' perception of their capability of understanding of the course content differed between the experimental and control group. A significance level was set as = 0.05.

Students' performance
We evaluate the students' performance in ISD, BI, and MIT courses by employing an independent samples t-test to compare the students' final scores. The mean scores of the students in the experimental and the control group showed significant differences. Overall, the analysis showed that the experimental group's mean score ( = 79.873, = 4.559; = 77.096, = 13.845) was higher than the control group ( = 72.807, = 10.363; = 69.593, = 12.451) at the 0.05 level of significance. For ISD course = 4.338, > , while BI-MIT course has = 2.094, > . These results confirmed the hypothesis ( ) that the experimental group result is significantly higher on students' learning performance when compared to the control group in information technology courses. It means that FL makes students performed academically better than traditional learning. Table 2 presents the summary of the results.  We also determined the students' grade distribution based on their final scores. In this section, we compared the students' final scores on ISD, BI, and MIT courses between experimental (n=77)and control group (n=75). From Figure 1 below, we can see whetherthe experimental group or control group has varying values. Based on its courses, most of the students in the experimental group on the ISD course has A/B and B-grade with a total of 10 students (20%). Meanwhile, in the control group, the highest percentage of students was found on the A/B grade of 35 students (72%).
Different things happen in BI and MIT courses. The course showed that most grades got by the control group was A with 40 percent of students. However, the students who got grade F (failed) almost as high as the A. On the contrary, in the experimental group, the grades more distributed. The highest percentage of the students got B+. Nevertheless, less than 15 percent of students who got A.

Students' perception
Generally, the students had a positive response toward the FL model. They stated that they liked the FL because they can fulfil their curiosity and effectively understand the course content. Some of the course content is delivered using videos. Students stated that using the videos facilitated their understanding because the videos can be access anytime and anywhere. They also agreed that the FL is making them more active in discussion and learning session. The quizzes that held in the class were also fun. The teacher used multiple platforms and helped them understand the content more.
The students' perception was focused on students' confidence in their abilities to complete a particular task and their capability of understanding the course content material. When asked about their confidence to complete the task, 45 participants (71%) agreed that they exited to learn and understand the course content before the class begins. Students stated that they could prepare themselves before the class begin and ask the teacher for further. Up to 31 participants (49%) will explore the course content as a reference. They become more active in participating in every class session of FL class than in the traditional class. In the closed-ended question on the course content understanding section, up to 50 participants (77%) agree that they can understand the material and analyze each course and example. Furthermore, 44 participants (68%) state that they can do de quizzes, assignments, and tests easily.
We determine the FL's impact in improving students' confidence to complete the task and their capability to understand the course content by using a two-proportion z-test to compare them. Twentyseven percent of the experimental group, perceive that their confidence to complete the task is "Very High" ( = 0.995, > ). However, only 20 percent of participants in the control group sense the same condition. The results show that the students' confidence in the experimental group to complete the task was higher compared with the control group. Afterward, we also used a two-proportion z-test for analyzing students' capability to understand the course content. As shown in Table 3., 11 participants categorized as "Very High" on students' capability to understand the course content ( = 0.823, > ). Nevertheless, only 7 participants categorized as "Very High" on the control group. The results show that the students' capability to understand the course content was higher compared with the control group.
These results confirmed the second hypothesis ( ) that students' perception of the experimental group was higher when compared to the control group in information technology courses.

Discussion
The results of this study show that the students in the FL increased students' performance and perception of information technology courses. Regarding students' performance, scores of the students who used the FL model were higher than the students who used the traditional learning model. This result cannot be separated from the fact that the students learn the course content before the class begins. They learn the content material and looking for a similar reference for their understanding. The students were also actively asking the questions to the instructor in the classroom session. They receive immediate feedback from the instructor and have much interaction with their friends and instructor during a lesson. The students came to the class more prepared and it increased the student-student and the student-instructor interaction [10], [16], [18], [21], [22]. In terms of the students' performance, [8], [10], [15], [22] showed the improvement. However, the other findings have shown that the FL model has a less positive impact on students' performance [3], [13], [23]. The differences could be related to the various materials, the classroom environments, and the processes used to apply and support the FL model. The types of the materials used and the surrounding learning environment also can affect students' performance.
Based on the open-ended questions section, student response positively toward the FL model. Students stated that FL supported their learning process when it comes to practicing section. They loved to involve and remain to be motivated in every learning process. It is easier for them to apply the concept that they have learned before. Students had access to learning resources at their own pace whenever they needed it. This approach ensures that students can study and understand the course content on their own. This study confirms the other studies, such as [2], [13], [21]. The students also stated that the FL change their old learning habits, and they enjoyed it because they could teach themselves before class and discuss with others in the class, looking for answers to question actively. Unfortunately, some students find that discussion is hard if the class environment does not support it. It might be easier for them to know what will they learn over the next six months.
Some of earlier research studies showed that the FL model was more suitable for upper educational level [23], [24], as the FL model may be hard and challenging for students who have not developed strong study skills [13], [24]. Also, the FL model may be more suitable for a theoretical course. It happens because students learn and understand the theoretical part easier [7], [13]. There is a lack of earlier research on the applicability of the FL model and suggest future studies should investigate this research gap. This study reports the success of implementing the FL in technology course that more practical.

Conclusion
This study found some promising significant impact of implementing the FL model on information technology courses. The improvement of students' performance is proof that the FL model design appears to make a difference in the educational field. By using the FL model, students can spend enough time watching video lectures, preparing themselves well before attending the class (face-to-face) session in which they are actively involved in solving problems based on the questions given. In terms of students' perception, the FL model results in more significant changes in the students' abilities to complete a specific task and their capability of understanding the course content as compared to the traditional learning model. The study showed that the FL model also suitable for technology courses that have more practicing sections.
The FL effectively improves the students' performance and changes the students' perception in the learning process. However, the limitations of this study should be pointed out. Further research in the form of a further study and more practical courses may be useful in clarifying the other aspects that impact the student.