{'fraction_non_alphanumeric': 0.029334589248469354, 'fraction_numerical': 0.012447016080199153, 'mean_word_length': 4.634571645185747, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Usability and User Experience Test on the Muezzin and Khatib Scheduling Mobile Application Using SUS and User Experience Questionnaire (Case Study: Al Abroor Mosque)

Received April 23,, 2020 Revised June 3

One of the Dewan Kemakmuran Masjid (committee to serve public service in a mosque) main task is to provide prayer facilities in a mosque, such as managing schedules for muezzin and imam salat. Most of the time, the plan is executed manually. Moreover, preparing the schedule is not easy. It needs cautious and excessive effort as DKM members should be careful not to make an unbalanced schedule for each muezzin/ khatib. It is also an error-prone process since putting the unavailable person in the schedule slot is possible. To overcome the condition, we build a mobile application for managing schedules for muezzin and imam salat. The application has been tested with usability testing and a user experience questionnaire. The results from both testing tools yield excellent results; the application could implement and comfortably used for managing schedules. From the result, it can be inferred that Al Aboor meets the criteria for a good mobile application and can be developed further.

Introduction
One of the Dewan Kemakmuran Masjid (DKM), a committee to serve public service in a mosque, tasks is to provide prayer facilities in a mosque and manages the schedule for muezzin and imam salat. The plan is usually executed manually; DKM holds a meeting to discuss and allocate each person. After that, DKM should ensure that the selected person is accomplished with their respective duty. Preparing the schedule is not simple; it needs cautious and excessive effort as DKM members should be careful not to make an unbalanced schedule for each muezzin/ khatib. It is also an error-prone process since putting the unavailable person in the schedule slot is possible.
Furthermore, it is possible that the person in charge does not make it up to their task or could not accomplish the duty because of some circumstances. When these problems arise, DKM should find another person to replace the duties person. It is troublesome since finding a replacement can be quite hard. Usually, others have their schedules packed, especially if there is limited time to find the substitute.
Nowadays, several mobile applications have been developed to help scheduling in a mosque. Maulid et al. developed a collaborative-android based system for Majlis Tabligh Muhammadiyah Bandung. It manages and displays the annual Jumu'ah sermon schedule for khatib at 21 mosques in this region [1]. A web-based system for Jumu'ah sermon in Makassar was presented in [2]. Faulina utilizes a mobile web framework to develop a scheduling application for Jumu'ah and daily prayer custodians [3].
Al Abroor is a mobile application that helps DKM arrange the schedule for muezzin and imam for daily prayer at Al Abroor mosque. It also has a scheduling feature for imam and preacher for tarawih prayer at Ramadan and some other features [4]. The Al Abroor mobile application makes it easy for committees to serve public services in the mosque to determine, change, and synchronize the muazzin and khatib schedule to minimize error-prone. Al Abroor was initially developed in a narrow focus, i.e., for Al Abroor mosque in Banjaran, Bandung. However, it can be expanded to accommodate other mosques in other areas. Before expanding the application, measurement should be made to ensure that the application meets the criteria for an excellent mobile application.
One prominent factor that should be taken into consideration is the usability aspect of the app. It is crucial in determining a mobile application's success rate and user acceptance [5]. It also shows the quality of an application. A proper application, or software in general, should fulfill usability aspects that are easy to learn, efficient to use and meet user satisfaction [6]. Usability is a good method to measure the quality of an application. However, sometimes it is necessary to measure other aspects than usability, and user experience is usually used in conjunction with usability [7]. User experience is used to measure whether an information system/ information technology gains adequate acceptance from its users [8].
Usability and user experience testing for Al Abroor mobile applications is present in this paper. Both measurements are done to ensure the quality of the app before further development. The remainder of the paper is organized as follows. Chapter two describes Al Abroor, the software under test. Section three explains the methods used in this paper, while chapter four discusses the result of the experiments-finally, the conclusion of this work is present in chapter five.

Al Abroor Mobile Application
Al Abroor is the software under test in this paper. It has several features: schedule display of muezzin and imam for daily prayers and Ramadan, events held in the mosque, list of DKM committees, prayer times, qiblah direction, and information about the application as well as its developer that shown in Figure 1.

Figure 1 Al Abroor Main Menu [4]
Among those features, scheduling and displaying events are the main features of this application. Muezzin and imam schedules are arranged by DKM and can be shown by clicking the respective date on the calendar ( Figure 2). DKM member is also responsible for filling in upcoming events that will be displayed in a list of events, as shown in Figure 3.

Methods
Two different approaches are utilized in this study. System Usability Scale (SUS) measures the usability aspect of the Al Abroor mobile application, whilst the User Experience Questionnaire (UEQ) is used to measure its user experience. SUS is often depicted as a "quick and dirty method" to measure a system/ application [9]. It has been used for many systems and referred to as an industry standard by some researchers [10]. UEQ is developed to be a quick assessment questionnaire done by end-users. It is claimed to be covering a "comprehensive impression of user experience." It aims to let users express their feelings, impressions, and attitudes towards the software under test simply and directly [11].

Measures
Participants for this study are selected among DKM Al Abroor members and residents in the Al Abroor neighborhood. There are 16 and 20 participants for SUS and UEQ tests, respectively. The number is picked based on the minimum data collecting requirement for each method. It is suggested to collect quite a large number of data since it will result in a more accurate conclusion that can be drawn from it. However, 20-30 participants are suitable to give a stable result for UEQ [12].
Meanwhile, SUS can be conducted with relatively few participants. Around 10 participants could give a suitable result for this usability testing, and it can give a relatively correct conclusion over the software with only 12 participants [13]. For example, research performs SUS testing for its web application with only five respondents, and it gets a good result despite the small number of participants [14].
The study is conducted by enquiring about the participant using and exploring all Al Abroor application features. It will take just a few minutes since the feature in this application does not require complex action to accomplish. After finishing investigating the app, the participant is asked to fill in the questionnaire. Data collected by these questionnaires will be analyzed using SUS and UEQ methods as described in subchapters 3.2 and 3.3.

Figure 3
Event menu, (a) The list (b) Input menu for event [4] 3.2. System Usability Scale (SUS) SUS consists of 10 questions that cover various aspects of system usability. Each question has an individual score, from range 1 (Strongly disagree) to 5 (Strongly agree) [9]. Initially, it was created for English-native speakers, but it expanded to non-native English countries, with a translated version of SUS has been created in those countries [10].
Sharfina and Santoso [15] propose an Indonesian adaptation of SUS. It has a similar semantic with the English version and is believed to have no grammatical difficulty. It has been validated with the respondents and is considered reliable due to its acceptable Cronbach's Alpha test [15].
The SUS score ranges between 0 and 100. Each item contributes to the overall rating; therefore, a respondent should mark all questions. However, there are different measurement for the items. The score for odd items (items 1, 3, 5, 7, and 9) is calculated by its scale position minus 1. The formula for calculating the score for the odd item is:   Table 1 sample (for simplicity, this example includes only five questions),
 Score for item 1: 3 -1 = 2  Score for item 2: 5 -5 = 0  Score for item 3: 4 -1 = 3  Score for item 4: 5 -2 = 3  Score for item 5: 4 -1 = 3 The average score for SUS is 68; it is considered as the 50th percentile in the SUS curve (Figure 4). Sauro proposes the percentile graph, and it can be used to interpret SUS raw score. The graph is devided the score for grade F to A, with the score below 51 represents an F grade, and score above 79 is positioned in the Arange [16].

Figure 4
Percentile graphics rank towards SUS Score [16] 3.3. User Experience Questionnaire (UEQ) UEQ takes into consideration both pragmatic and hedonic aspects [11]. It comprises 26 questions in six scales of user experiences, attractiveness, clarity, efficiency, dependability, stimulation, and novelty. Attractiveness distributes in 6 items, and other scales are in 4 items [12]. Attractiveness measures the overall impression of the product, whether the user likes it or not. Perspicuity measures the easiness of getting familiar with the product. Efficiency evaluates whether a user can solve the tasks with the product without unnecessary effort. Dependability evaluates whether the user feels in control of the interaction or not. Stimulation shows the excitement and motivation in using the product, novelty measures the innovative and creative aspect of the product [17].
The original version of UEQ is in Dutch. However, its version in other languages has been developed, and some are validated [17]. These versions are available at www.ueq-online.org [12]. An example of UEQ questions shows in Figure 5.

Figure 5
Sample of UEQ Questions [12] Any standard statistical tool can measure each UEQ question to get means and deviation for the six scales (attractiveness, efficiency, perspicuity, dependability, stimulation, and novelty). However, UEQ is equipped with a standard measurement tool that can be downloaded from its site. The tool generates means for each scale from the answered questions; the range for UEQ's scale is distributed from -3 (terribly bad) to 3 (extremely good). In standard interpretation, values between -0.8 and 0.8 are considered neutral results, whilst values >0.8 refer to positive results and values < -0.8 refer to negative results. The provided tools represent the value in the graphical picture; therefore, it will be easier for the reader to comprehend the result [18].
UEQ can compare user experience for two or more products, but it can also be used with a single product. In measuring a single product, UEQ provides a benchmark to determine whether the product is an acceptable user experience. The benchmark states five criteria range from excellent to bad for the six UEQ scales [17]. Table 2 shows the benchmark intervals.

Result and Discussion
This section will show the result of the software usability testing (using SUS) and UEQ testing. Both of the testing tools show good results. The SUS tool shows the result is above the average score, and the UEQ testing tool shows a good result considering factors such as the mean scale and its benchmark. The detail for the explanation of the result will describe in the following section; 4.1 for the SUS and section 4.2 for the UEQ.

SUS Results and Discussion
Based on the survey calculation technique explained in section 3.2, the calculation gets a score of 85.6. The 85.6 score means the usability of the application has a grade of A. The grade A is determined from the graphics in Figure  4.
As shown in Figure 4, the SUS graphic mentions the application has good usability if it has a score above 70 or has a grade B minimum. Al Abroor, the application under test, gets a score of 85.6, and thus, as mentioned in Figure 5, has an A grade. The grade A also indicates that the user is comfortable using the app and will recommend it to other users.

UEQ Results and Discussion
UEQ tools testing, as mentioned in section 3.3, is used for evaluating the interactivity of the application. UEQ has 26 questions for respondents to fill in. The questionnaire was distributed to 20 respondents, and the answers from the respondents are shown in Figure 6.

Figure 6 The Respondents' Answers
Data collected (as shown in Figure 6) is further converted to obtain the value (scale one to seven) that can be calculated to show the results that inform the application's interactivity. The conversion of value in Figure 6 is shown in Figure  7.
The result is shown in Figure 8. The figure implies that the application could fulfill the user's expectations. The mean value for attractiveness, perspicuity, efficiency, dependability, stimulation, and novelty of the application displays a good result with value for all scales around or more than +2. The prominent feature of this application, as can be inferred from the result, is that it can fulfill its users' expectations.

Figure 8
Mean of Impression Figure 9 shows the result compared to the benchmark; this graph is also generated by the provided tool of UEQ. The figure supports the previous result that the application has fulfilled the user expectation. The graphics result acknowledges that the application has above the average score, and no score is under 1.0. Referring to Table 2 of Benchmark scale for UEQ, the application has excellent result.

Conclusions
Al Abroor is an application that can be used to arrange muezzin and khatib scheduling at a mosque. This application is utilized to help DKM manage the schedule for muezzin, imam, and events at the mosque. SUS is utilized as a usability measurement for the application, and it gives a good result with an 85.6 score (grade A). The usability score mentions that the application could implement scheduling muezzin and imam salat at Al Abroor mosque in Banjaran, Bandung. In conjunction with usability testing, the user experience of this application is also measured with UEQ. UEQ testing has an excellent result. It indicates that the application has a good impression on the user, and the user feels comfortable when using it. It can be inferred that Al Aboor meets the criteria for a good mobile application and is sufficient to be further developed.