{'fraction_non_alphanumeric': 0.020578545963265492, 'fraction_numerical': 0.008168238638927749, 'mean_word_length': 5.003793010024384, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Problem-based Learning Laboratory (PBLab): Facilitators' Perspective on Rubric Assessment

Problem-based Learning Laboratory (PBLab) has been established as one of the core courses for a Bachelor of Engineering (Electrical) program at Universiti Teknologi Malaysia (UTM) since 2007. Several initiatives have been made in reviewing the PBLab evaluation criteria to ensure that the students are properly assessed and their ability to acquire both technical and generic skills can be successfully measured. General analysis of the students' grades for the past few years have revealed a trend of inconsistency between that obtained from the PBLab with those obtained in other courses. This paper attempts to verify this inconsistency while at the same time focuses on how the students are being assessed by the PBLab facilitators using the assessment rubrics as a possible factor contributing to this inconsistency. The former has been verified through the study and the analysis made has revealed that the latter is highly influential in determining the students performance in the PBLab, thus becomes a contributing factor to the inconsistency.


Introduction
Problem-based Learning Laboratory (PBLab) is a 4 th Year Undergraduate Laboratory course that has been implemented at the Faculty of Electrical Engineering, Universiti Teknologi Malaysia (UTM) since 2007. It has been identified as an effective student centred learning approach used in conducting a laboratory which provides the students the experience of solving and analysing real world or close to real world problems while conducting relevant experiments (N. A. Azli, 2010). These experiments can be hardware based, software based or both depending on the area of studies focused by the laboratory concerned.
Engineering educators in today's world are pressured to produce graduates who are not only excellent in knowledge, but also in the acquirement of soft skills as required by the industry such as critical thinking, multidisciplinary, team-player and effective communicator (Khairiyah Mohd Yusof, 2004). In answering the demands of the industry, the PBLab has been initiated since 2004 in terms of design and planning until its actual commencement in 2007. Although proper planning has been made for the PBLab prior to its implementation, the performance of the students who underwent the course has constantly been assessed, not only through the final grades obtained by them but also on their level of achievement of the course outcomes. Feedbacks obtained from the students on the aspect of team working, laboratory conditions and problems assigned have been used as continual quality improvement of the laboratory. This paper presents the results of an analysis made on the grades obtained by the students for the PBLab, in comparison to that obtained from another core Electrical Engineering course. This is done to determine the correlation between the two on the aspect of the students' performance. More importantly this paper attempts to highlight the perception of the PBLab facilitators towards the assessment rubrics that has been designed for the laboratory. This is vital because the assessment rubrics are the main component that determines the students' grades. By understanding the facilitators' perceptions on the rubrics, further improvement can be made to ensure that the students are being properly assessed by the facilitators thus reflecting their true ability. The next section describes the structure of the PBLab. Then some background on the evaluation criteria that have been designed for the laboratory is elaborated followed by the methodology of the study. The results are then presented with the relevant discussion before concluding the study in the final section.

Problem-based Laboratory (PBLab) Structure
PBLab for an undergraduate Electrical Engineering program has been developed and successfully implemented since 2007. Its aim is to induce some changes in laboratory conduct from teacher-centred to student-centred learning approach. Based on a particular student cohort (N.A. Azli,2005), the PBLab has been highlighted as a course that can encourage students to be good team players and creative problem solvers in facing real world issues in the laboratory. Students' feedback as extracted from the Peer and Self Evaluation forms during the semester 2008/2009/2 have revealed that PBLab benefits the students greatly especially in improving the technical and generic skills for their future workplace (N. A. Azli,2010). This paper focuses on the PBLab implementation in a Bachelor of Engineering (Electrical) program at UTM which is better known as the SEE program. Table 1 gives some description of the PBLab.

4.
Time allocation a) Each group is given 3 hours per week in-lab session with facilitation and 2 hours per week out-lab session with group members outside of the laboratory. b) Each group is required to solve 3 different problems or projects (1 from each laboratory)by conducting experiments, within 4 weeks.
The PBLab structure as shown in Table 1 is divided into 4 main categories which are the students' level, grouping, laboratories and time allocation. All of these categories are related to each other in ensuring successful implementation of the PBLab. For the students to get a clearer picture on the PBLab learning approach, the activities of the laboratory from week 1 until 4 are given as stated in Table 2. 1. Continuation of report writing and submission exactly one week later to the Lab technician to be recorded and given to facilitators. Table 2 describes the facilitators' responsibilities in each week of the PBLab sessions. It indicates the importance of facilitation in ensuring the smooth conduct of the PBLab activities within the time allocated. Generally, facilitation in problem-based course means helping students to learn to trust their own decision-making and problem-solving skills and to give up accepting others decisions about what is relevant or true (Christine Alavi,1995) . For the PBLab course, the roles of the facilitators have been stated as follows (N.A.Azli, 2010): 1. To facilitate each group in a laboratory session in solving a problem or conducting a project. 2. To evaluate the student's laboratory performance based on the outlined evaluation criteria. 3. To make sure that the evaluation process is completed according to schedule for each assigned problem/project.
In general, facilitation in a course that adopts the Problem-based Learning (PBL) approach can help the students identify and rectify the difficulties which occur in the learning process (Christine Alavi,1995).Thus, in order to make sure the PBLab objective which is to produce active learners among students is achieved, it is essential for the lecturers who are the facilitators of the PBLab to take full responsibilities in ensuring that the students are properly and effectively assessed based on their performance. Good assessment criteria have been designed to help the PBLab facilitators assess the students' performance in the laboratory fairly.

PBLab Evaluation Criteria
Rubric assessment has been selected by the PBLab Task Force members to be one of the evaluation tools used to assess the students' performance (N.A. Azli, 2005). This is in line with the fact that rubrics are the main assessment tool that has typically been used by facilitators to evaluate students' performance under most of PBL criteria (Heidi Andrade and Ying Du, 2005). Table 3 shows the evaluation criteria of the PBLab.

5.
Group report 20 Total 100 Based on Table 3, for criteria 1, 3, 4 and 5, assessment rubrics have been developed to allow the PBLab facilitators to rate the students' performance, whether as an individual or as a group. Ratings are set to be in the range of 1 to 4 with 1 rated as poor followed by fair, good and excellent for each of the components under each criteria. This evaluation process is repeated when the group of students moves to other laboratories in the 5 th and 9 th week of the PBLab sessions (N.A. Azli, 2010). Therefore, it can be seen that rubric assessment in the PBLab is very important to indicate the students' performance each week. Well developed rubrics should be made available in order to make the facilitators' expectations clear while assessing the students' achievement. However, initial analysis of the PBLab grades obtained by the students based on the available rubrics has indicated inconsistency with the grades obtained by them in other courses. Hence, there is a possibility that the true ability of the students in achieving both technical and generic skills from the PBLab has not been properly assessed using the rubrics. Several factors may contribute to this issue and the way the PBLab facilitators use the assessment rubrics and their perception in evaluating students will be highlighted further in this paper.

Participants
Twenty final year undergraduate students from the Faculty of Electrical Engineering, UTM have been selected to participate in this study. All of them are students of the SEE program that include twelve males and eight females. For this study, it is essential for these students to have undergone the Power Electronics course which is a core course for the program other than the PBLab course itself. Besides, three PBLab facilitators also have been chosen to be interviewed in order to get their perceptions towards the PBLab rubric assessment.

Procedures
This study that basically uses sequential explanatory strategies begins with the analysis of quantitative data using Predictive Analytics Software (PASW) followed by face-to-face interviews with a few selected PBLab facilitators. Specifically, this study utilizes the Paired-Sample T-test quantitative approach that involves the measurement of two variables which are the students' grades for the Power Electronics course and the PBLab course particularly based on the assessment made at the Power Electronics Laboratory. The students' grade data has been used to provide solid evidence on the issue related to its inconsistency. Then, to support the results obtained from the quantitative data, interview sessions with the selected facilitators have been conducted in order to get their opinions on this issue.

Results and Discussion
To analyze the students' grades for both courses which are the Power Electronics course and the Power Electronics PBLab, a Paired-Samples T-test is conducted. This allows comparison on the students' grades for both courses. This test has been chosen due to the fact that in this case a group of the same SEE program students has undergone two different Power Electronics related courses (class and laboratory). Table 4 and 5 shows that there is a significant difference in the students' grades for the Power Electronics course (M=62.55,SD=7.37) and the Power Electronics PBLab (M=76.39, SD=4.25); t(19)=8.214,p=0.000. The significant difference between both students' grades can be proven based on the value of Sig. (2-tailed) in Table 5 which is 0.000. This value is less than 0.05. Therefore, it can be concluded that there is statistically a significant difference in the mean of the students' grades in the Power Electronics PBLab as compared to that in the Power Electronics course.
Since the Paired Samples Statistics results in Table 4 have revealed that the mean of the students' grades for the PBLab is greater than that of the lecture-based teaching approach Power Electronics course, it is evident that the former is capable of providing a better effect in terms of students grades compared to the latter. Students can score higher with better understanding when learning actively through the PBLab compared to passively listening to the lectures in class. However, the important issue that is highlighted in this paper is the inconsistency of the Power Electronics PBLab students' grades with that of the Power electronics course. It is expected that if the Power Electronics PBLab students' grades are high, the Power Electronics course students' grades must also be consistent to this. Generally, several factors can influence the students' grades between both courses for example types of assessment and evaluation, types of teaching approach and the way the students are being assessed by the facilitators in the course. However, a possible factor which is further investigated in this study is the manner of which the students have been assessed by the PBLab facilitators using the assessment rubrics.
Rubrics are one of the common tools that is used in assessing students' achievement and progress in various class activities. Steven and Levi (2005) have also reported that rubrics can improve consistency in the grading of students. In the case of the Power Electronics PBLab, the assessment rubrics play a big role in determining the students' grades. As highlighted earlier, the students' grades for the PBLab are higher than that of the Power Electronics course which do not seem to reflect the students' true knowledge on the subject matter. This brings about to certain issues that can be made the factors in influencing the facilitators when assessing the students using the rubrics. To identify the issues, face-to-face interview sessions have been conducted in order to get some opinions from the PBLab facilitators. The open-ended responses provide more in-depth perspective on the PBLab facilitators' perceptions on how the students are being assessed by them during the laboratory sessions. Whereas, the open-ended comments reflect the way the PBLab facilitators assess the students in the laboratory.
The first issue that has been highlighted based on the interviews conducted is the facilitators' emotions when assessing the students. Based on the responses obtained, most of the respondents tend to be sympathetic towards the students when assessing them. However, the rubric criteria have guided them in giving the marks not based on sympathy but more on the students' progress in solving the given problems. All the facilitators' said that the marks given depend on the students' performance themselves.  Based on Table 6, which is the statement regarding the facilitators' emotions while evaluating the students, there are three important key points that have been stressed out by the respondents which are "sympathetic", "performance" and "achievement". The sympathetic feelings are natural human feelings and people who have this positive emotions lead pattern of thought that are more flexible and linear (Tugade, 2004). However, these flexible and linear emotions have to be minimized to avoid the biasing issues. So, the most effective way to reduce it is by assessing the students based on the rubric criteria that have been stated. As has been mentioned in Table 6, both respondents have generally used the rubric criteria as a guideline to assess the students based on their performance and achievement in the laboratory. This can reduce the biasing effect in the students' marks. Unfortunately, although the rubric criteria have been used as a guideline by the PBLab facilitators, most of them do not clearly know how to define and differentiate the PBLab rubric ranking which is between 1 (Poor) and 4 (Excellent). Table 7 shows the feedbacks received from the PBLab facilitators on this second issue.  Table 7 describes another challenge that is faced by the PBLab facilitators which is their marking has not been truly guided specifically by the rubric criteria. The facilitators seem to think that they have already given the best marks to the students but the fact is they may not have done so. In addition, most of the PBLab facilitators' comments as stated in Table 7 indicates that the students have been marked based on the former personal assumptions. Furthermore, a few important words that have been highlighted in the comments above are "punctuality" and "dressing/dress code". Respondents A and C are clear on the punctuality and dress code criteria in the laboratory. They have mentioned this clearly and they are aware of the criteria. However, it is rather worrying if the students' marks have been given based on the halo effect. The halo effect is the condition when a facilitator tends to attribute additional positive characteristics to someone who has one salient quality such as physical attractiveness. Besides, this effect also occurs when the facilitators or lecturers know which student wrote which response and alter the grading depending on his or her perception of the student (Roxana Moreno, 2010).
Table 7 also shows that respondents A and B are more inclined towards assumption and observation. They do not mention that the students' marks are given based on the rubric criteria as have been mentioned by respondent C. Respondent C repeatedly mentions the word "criteria" which indicates his or her understanding on the importance of following the rubric criteria in assessing the students. Again, it is rather disturbing to know that respondent A and B tend to assess the students' based on their opinion and perception in the sense that "whoever does more work and performs better in the laboratory is eligible to get full marks". Generally, much research has shown that facilitators or lecturers' expectations may inadvertently create unfair practices (Graham & Barker, 1996). Besides, Roxana Moreno (2010) also stated that "it is unfair to reward students who do more than the work required and penalize with partial credit those who followed the instructions carefully and decided to not do more than what was required".
In order to avoid or minimize the biasing and fairness issues in the PBLab assessment, several improvements can be made. The first issue is the perception among facilitators towards the rubrics assessment. The facilitators need to be trained or explained in order for them to understand the differences of each rubric ranking that has been stated and how to judge the students based on their performance, appropriateness of attitudes and their knowledge development. It is recommended that in the early stages of the PBLab course, the facilitators need to inform the students on how their responses or the information they provide will be judged or scored. This can lead to the assurance that similar expectations are held by both the students and their facilitators (Joint Committee on Testing Practice, 1993). Frederiksen (1984) also reported that most students are more motivated to learn when they are being told that their learning will be assessed compared to when they are being told to learn the material. Besides that, care should also be taken to ensure the students' results are not influenced by factors that are not relevant to the objective of the assessment (Joint Committee on Testing Practice, 1993). The halo effect that has been mentioned earlier can be a a major factor influencing the students' marks. For this reason, it is suggested that the results are obtained from two or more independent raters such as other facilitators and students which will generally produce a more consistent description of the students' performance rather than that obtained from a single rater.
In addition, other improvements or modifications that need to be made is the rubric criteria itself. This is based on the respondents' comments during the interview session as given in Table 8.

Respondent B:
The rubrics are more for group marks. Not much for individual marks. Surely for PBLab we want to look at team working, but individuals can't just become passengers.

Respondent C:
Perhaps supposedly, there are individual marks for the presentation part.
The PBLab assessment rubrics seem to be focusing more on group work rather than individual work. For example, the logbook, presentation and report are assessed based on group work whereas there is only one individual assessment which is that of the individual in-lab activities. Therefore, further study need to be conducted in determining the actual objectives of the PBLab course whether to improve the group working skills among the Electrical Engineering students or to enhance both individual and group working skills. This is because the type of assessment used will finally lead to the ability of the students to achieve the PBLab course outcomes. For this reason, it is important to ensure that the assessment process or criteria is carried out fairly and efficiently and at the same time reflecting the students' knowledge and skill acquirement from the PBLab.

Conclusions
To sum up, there are two issues that have been highlighted in this paper based on the study conducted on the PBLab. This includes the inconsistency of the students' grades and how the facilitators have used the assessment rubrics to evaluate the students. Analysis of the students' grades for both Power Electronics PBLab and Power Electronics course has shown inconsistency. There are many factors that can influence the inconsistency in the students' grades between both courses. However, this paper only focuses on the manner of which the students have been assessed by the PBLab facilitators using the assessment rubrics. Further analysis on the responds of the PBLab facilitators have been made and reveals that such inconsistency has happened due to three major factors which are misconception of the facilitators' perceptions towards rubric ranking, halo effect issues in the assessment and the objective of the rubrics. These three factors can lead to the inconsistency of the students' marks for the two courses particularly since rubric assessment in the Power Electronics PBLab is mainly group based. Investigation on the PBLab facilitator's answers and perceptions towards the PBLab assessment rubrics suggests that a clear guide on how to use the rubric ranking should be exposed to the PBLab facilitators due to the misconception issues. Moreover, a review of the rubric criteria has to be made in order to ensure that the marks given by facilitators' truly reflect the students' ability in terms of the expected knowledge and skills acquired as outlined in the PBLab course outcomes.