{'fraction_non_alphanumeric': 0.043518199563669764, 'fraction_numerical': 0.013893673211620163, 'mean_word_length': 5.029421945309796, 'pattern_counts': {'":': 0, '<': 2, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Building a Descriptive Exam Model for Student Performance using Process Mining Techniques in Examaize System

Paper The objective of this research paper is to determine the understanding of students’ course contents and evaluate student performance according to expectations. There are predefined navigation models available to answer Multiple Choice Questions (MCQs). The individual student responds to these questions in their navigation pattern that results in various Student Performance Outcomes. The research presents the process mining techniques to evaluate students’ answers in the MCQs predefined navigation model in the exaMAIZE Assessment System. This research shows that students navigate all the twenty-five questions according to their way of answering the exam, where seven questions are under high frequency and later analyze to revise the answer on suspicion or difficult. The experiment compared Student Exam Outcomes with the Student Performance Outcomes of each outcome, and the frequency of navigating a question to find out individual student Strengths and Weaknesses. This work can be extended future development of efficient process mining algorithms by applying the next academic session to improve student performance


I. INTRODUCTION
In Multiple Choice Questions (MCQs) exam, students navigate from one question to other by clicking the next button, skip the questions for difficult to answer, look up for the hint, change the answer by clicking the review question button and finish the exam [3,4,5]. Students' such skills and answering style creates different types of navigation patterns. Individual students' skill helps a different way to answer the MCQs exam and various navigation patterns generated. It is a great challenge to understand the multiple navigation patterns, and the researcher focused on enhancing the evaluation process of student performance [33,34].
Process mining techniques are based on data mining techniques where process mining is the set of methods that allow the extraction of process details, derived from a set of recorded real execution of the process (logs) [16,25,26]. The main aim of carrying out process mining is to gain insight into processes; the mined result needs to be presented to the student in a manner that can be understood. This technique helps to compare between conventional process model and the actual process model. The objectives are to find discrepancies to improve the process according to expectations. Some of the process mining algorithms evolved are an alpha miner, FSM miner, fuzzy miner, heuristics miner, multi-phase miner, genetic process miner, and region miner [7,30,31].
Weijters developed a process mining algorithm named as Heuristic Miner, which is used as a heuristic approach to address many problems, closely associated with the alpha algorithm. It can abstract from unusual behavior and noise (by leaving out edges) and, therefore, suitable for many real-life logs [10,32]. The researcher used the extracted data from the event log of the course COIS 481: Introduction to E-commerce exam conducted at the Department of Information Systems, Faculty of Computing and Information Technology in Rabigh (FCITR), King Abdulaziz University (KAU) by using the exaMAIZE assessment system.
The present research paper is organized in the following manner. In Section 2, the researcher discussed the use of process mining techniques to create a process model in FCITR's exaMAIZE Assessment System. In Section 3, introduced the numerical study and discussed on student performance findings. Finally, it provided conclusive remarks in Section 4.

II. RESEARCH METHODOLOGY
In this section, the researcher discussed on use of process mining in modeling and analysis, heuristic miner algorithm, conformance checking, and navigational model on exaMAIZE assessment system. In this part, the researcher used heuristic miner techniques to create a process model in FCITR's exaMAIZE Assessment System.

A. Process Mining
In the last decade, process mining has emerged as new research which focuses on extracting information from an event log recorded in a BPM / WFM system [8,29]. For process modeling and analysis, this technique is a discipline in the form of a combination of intelligence and data mining [9].  [14]. These include; (a) Process model discovery, (b) conformance checking, and (c) enhancement. The researcher utilized process mining techniques to discover a model of the FCITR students' navigation pattern in the exaMAIZE assessment process that results in different exam performance and results to improve the online evaluation process.

B. Heuristic Miner
The heuristic miner is a stable process mining algorithm that is well-implemented in many cases [6]. This algorithm implemented for the process mining task. Specifically, the discovery task, modeled based on the originator (actor/student) [10]. It helps to define the pattern frequency of the event logs that will be most appropriate to describe with the heuristic miner.

C. Conformance Checking
Conformance checking refers to a business process, compliance, and runtime [18,27,35]. The existing process models are compared with actual observations in conformance checking to assess the quality of the process [15,17,28]. Focusing on measuring and ensuring the majority of the conformance checking ensures that models catch all the behaviors stored in logs, i.e., fitness [13,15]. The following Fig 2 shows the fitness formula at the log level.

Fig. 2. Formula of fitness
The processes keep on continuous changing; conformance techniques are essential to identify obsolete or deficient models. There is also a need to compare multiple models for the same process and to determine the best option, for example, between models derived from different discovery algorithms.

D. Event logs
Event logs (i.e., conformance checking) are the starting point of process mining [19,20]. This event logs help to check navigation patterns correspond to the predefined navigation model. The analysis results may suggest improving the exaMAIZE Assessment System process to reflect the individual student's performance. As already discussed in the heuristic miner, which focused on the control-flow perspective to find the process instances and generated a process model in the form of a Heuristics Net for the given event log [10,21,22,23,24].

Fig. 3. FCITR's exaMAIZE assessment system E. exaMAIZE Assessment System
The exaMAIZE Assessment System is a web-based application developed with the latest Goal-Driven Service Oriented Architecture (GDSOA). The Department of Information Systems, FCITR, conducted a MCQs exam for the course material exam for COIS 481. This MCQs are in different forms like auto (system generated, i.e., intelligence), standard (prepared by instructor/faculty), ABET (Accreditation Board for Engineering and Technology), KAUS (Knowledge Application Understanding and Skill), RUBRICs form. The Bachelor of Science (Information Systems) degree program of FCITR adopted the ABET computer accreditation commission (CAC) from 'A' TO 'I' as "General Criteria" Student Outcomes (SOs) and that of Outcome 'J' recommended by the "Program Criteria".
This course material exam for COIS 481 consists of four criteria ('C', 'E', 'G', and 'J') as SOs of ABET standard of Department of Information System. Here, 'C', 'E', and 'G' comes under "General Criteria" whereas 'J' under the "Program Criteria".

F. Navigational Model on exaMAIZE Assessment System
According to Dr. Lal [11], in a web application, a navigation model has described a path to the individual student to communicate from one web page to another web page. In exaMAIZE Assessment System, navigation model defines as the students' interact the exam from one question to another by clicking the 'start', 'next', 'back', 'finish exam', 'review question', and 'hint' button [1,11,12].

III. MATERIALS AND METHODS
The researcher used heuristic miner techniques to create a process model in FCITR's exaMAIZE Assessment System. The input, output, and process of this research work explained in detail.

A. Input Analysis based on Exam Rules and Student Event Log
In the last FCITR Online exaMAIZE Student Exam Rule: Course instructors/faculties create a rule for the online course material exam for COIS 481.
FCITR Online exaMAIZE Student Event Log: The individual students' click data on the online course material exam for the COIS 481 exam conducted at FCITR.

B. Output Analysis based on Rule Conformance and Student Performance
Rule Conformance: Analysis of the navigation model of the individual student and the conformance to the predefined rule.
Student Performance: Student Exam Outcomes, Student Performance Outcomes, Student Strength and Weakness, and finally, the class discussion.

C. Process Analysis
At FCITR, ten students gave an online exam on Tuesday, 24 April 2019, for the online course material exam for COIS 481. In this FCITR Online Exam, 25 MCQs were asked. The Time Stamp, Student ID, students' actions from beginning to the end of the exam were noted. Student Exam Outcomes recorded in each step as ('C') for "Correct" answers, ('W') for "Wrong" answers, and ('S') for "Start", "Finish", and   The Process discovery performed based on the student navigation model and selection of questions. Conformance checking applied to derive the model process and calculate fitness value. In the end, the analysis and evaluation process to evaluate the Student Performance Outcomes and check the conformance of the process model.

IV. RESULTS AND DISCUSSION
In this section, data gathering and preprocessing techniques, process discovery phases, conformance checking, online course material student exam performance, and evaluation outcomes results are discussed.

A. Data Gathering
The researcher conducted an online exam for the course material exam for COIS 481 on Tuesday, 24 April 2019, on ten students. The data collected in two different forms.
1. The FCITR Online exaMAIZE Exam Rule consists of the following data: The The questions are answered from Question-1 to Question-25 by clicking the next button. The student can answer/skip the question in any order by clicking the 'start', 'next', 'back', and 'review' question button.
2. The Student Event Log consists of the following data: The sample of student event log data in Table I shows the Time Stamp (Date and Time), Student ID, Student Name, Action, and Result. The actions may be "Start", "Finish" or, "Question-1" to "Question-25" which depends on students' selection. The result is stored based on students' answer i.e., "Correct" for 'C', Wrong for 'W', or "Skip" for 'S'.

B. Preprocessing
During preprocessing, the Action is carried out from Data Log Type-(shown in Table II) as "Question Number", "Start", or "Finish". For Data Log Type-II (shown in Table III), gets ABET criteria Outcome Code ('C', 'E', 'G', and 'J') as Action.
This Data Log Type further converted into a newer format shown in Table IV. This table contains Case ID, Task ID

C. Process Discovery
The field of process mining focuses on the improvement of analysis, verification, and process models in the workflow system. ProM, an open-source framework used to create a process model using student event log data.
The Heuristic Miner plug-in used in two-phase process discovery to result, i.e., 1. Individual Students' Navigation Model (Process Discovery Phase-I), 2. Heuristic net of all students (Process Discovery Phase-II).

International Journal of Innovative Technology and Exploring Engineering (IJITEE)
ISSN: 2278-3075, Volume-9 Issue-3, January 2020 1. Individual Students' Navigation Model (Process Discovery Phase-I): Given below a sample example of student navigation produced from Process Discovery-I.

Fig. 5. Petri net of Process Discovery Phase-II
In this navigation model, the student navigates all the questions sequentially and later analyzes to revise the answer on suspect ion or difficult. The student visits again to read the question and verify the answer. The Task ID defines as the number of times (frequency) student visits the question number(s) shown in the depicted navigation model.
From Table V, student navigation frequency 14 signifies that question-4, question-11, question-12, and question-13 are most difficult to answer. The difficulty level of question-8 and question-22 are more as compared to question-17. Overall, in total, seven questions of navigation frequency are high.
2. Heuristic net of all students (Process Discovery Phase-II): Fig 5 shows the Petri net of Process Discovery Phase-II, where students answer the questions sequentially starting from Outcome 'C', 'E', 'G', to 'J'. Students complete the Outcome before going to another Outcome, i.e., in-Outcome loops of the dependency value are higher than moving to other Outcomes.

D. Conformance Checking
One of the process mining techniques known as conformance checking, which is used to compare the process model, resulted in the obtained student event logs with the Petri net. Petri nets act as intermediate steps, which are more straightforward to check the conformance of a simple formal model. The fitness value shows in Table VI. Fitness Value of Data Log Type-I is 0.81907834, indicating that 81.90% (i.e., fitness < 1.0) of the events in the log are possible according to the model. Similarly, Fitness Value of Data Log Type-II is 0.99804509(i.e., fitness < 1.0), indicating that 99.80% of the events in the log are possible according to the model.

E. Online Course Material Exam Evaluation
The online course material exam for COIS 481 exam conducted at FCITR and the Student Exam Outcomes calculated by multiplying the number of questions answered correctly with 4 to convert to 100% i.e. StudentExamOutcomes = TotalScoreCount(C) X 4 The student-1 received highest outcomes (90) whereas student-8 and student-9 received lowest outcomes (81) respectively. Student Performance Outcomes shows the results of an individual student based on their correct answer, Time Stamp, and student navigation to each question. The criteria for Student Performance Outcomes is 100% (High) if the student gives a correct answer between 0-1 minute. Similarly, Student Performance Outcomes is 75% (Medium) for 1-2 minutes. Further, Student Performance Outcomes is 50% (Low) for above 2 minutes. For, wrong or skip questions, Student Performance Outcomes is 0%. Outcomes, while student-2, student-7, and student-10 receives the highest Student Performance Outcomes. As a result, it shows that the student-1 gives correct answers at most, but higher confidence correct answers are given by student-2, student-7, and student-10, respectively.
To find out the student's Strengths and Weaknesses, the researcher compared

Fig. 7. Student Strength and Weakness in ABET criteria
Student Exam Outcomes with the Student Performance Outcomes of each Outcome, and the frequency of navigating the question. Table VII shows the student's strength and weakness Outcomes.
Here, the strength of the student derived from the low frequency with the highest Student Exam Outcomes, the highest Student Performance Outcomes. Similarly, the weakness of the student derived from the highest frequency with the lowest Student Exam Outcomes, the lowest Student Performance Outcomes. One can easily see in Fig 7, which shows that the FCITR students are strong enough in ABET criteria 'E', which refers to "An understanding of professional, ethical, legal, security and social issues and responsibilities" Outcome. Moreover, students are weak in ABET criteria 'G', which refers to "An ability to analyze the local and global impact of computing on individuals, organizations, and society" Outcome.
Data Log Type-I and Data Log Type-II from process discovery, Question-4 ('C'), Question-11 ('G'), Question-12 ('G'), and Question-13 ('G') are the most difficult as students navigated more. The Question-8 ('E'), and Question-22('J') are coming under the next difficult questions as their frequency is 13. The three most difficult questions are Question-11, Question-12, and Question-13, which are under ABET criteria 'G' Outcome.
The result of this evaluation on the conformance is that all students follow the predefined rules. All students navigate all the 25 questions sequentially, where seven questions are under high frequency to repeat and later analyze to revise the answer on suspect ion or difficult.

V. CONCLUSION
This research shows that students navigate all the 25 questions sequentially, where seven questions are under high frequency and later analyze to revise the answer on suspect ion or difficult. To find out individual student's Strengths and Weaknesses, the researcher compared Student Exam Outcomes with the Student Performance Outcomes of each outcome, and the frequency of navigating a question.
In this research, FCITR students are strong enough in ABET criteria 'E', which refers to "An understanding of professional, ethical, legal, security and social issues and responsibilities" Outcome. Moreover, students are weak in ABET criteria 'G', which refers to "An ability to analyze the local and global impact of computing on individuals, organizations, and society" Outcome.

ACKNOWLEDGMENT
The researcher would like to thank the King Abdulaziz University, Kingdom of Saudi Arabia and MAIZE Software Pvt. Ltd., India, to complete this research work.