{'fraction_non_alphanumeric': 0.020888171860405876, 'fraction_numerical': 0.007217457756644306, 'mean_word_length': 5.14397496087637, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
User requirement and usability testing framework for information system development: case study of financial institution

Due to the fact that financial institution has its limited competency in developing the information system, outsourcing is often used to support the information system development which same as the case study of the issuing documents system in financial institute. The financial institution has received customer’s complaints regarding the mistake of issuing documents caused by the system functionality and human error in operations. Thus, the purpose of this study is to design framework of user requirement and usability testing for information system development to prevent the error in system functionality and set some quality checking process of documents before delivery to customer to prevent mistakes from human error in work operation. System Usability Scale showed that after proposed the framework of user requirement and usability testing to employees who related with the system development, their level of satisfaction toward the framework and non-framework of user requirement and usability testing raise up to 30.93% and 20%, respectively. After six months of implemented quality checking process of documents revealed zero complaints from customers regarding to the mistake of issuing documents.


Introduction
It cannot be denied that the progression of technology has been an essential in current period. Information technology has been the crucial business resource and has assisted industries in terms of their opportunities, preferably bank and financial industries where the information technology plays an important role to reform financial institution to raise the speed and trustworthiness [1,2]. Financial institutions have developed a system to issue documents for customers regarding statement, invoice, and receipt, using a third-party company to support the process of system development. The process of document system development can be seen in figure 1. Most of the issues from the system development are often caused by the process of gathering the needs and testing [3]. After launching the document system, it was found that there were complaints regarding customer's financial statements, the first complain was incorrect financial balance statement that occurred on July and October accounted for 133,370 customers and 3,700 customers, respectively. The second complain was wrong document delivery occurred on August for 333,000 customers. The third complain, on September, was incorrect information in document accounted for 27,512 customers. The root cause finding and analysis was conducted using 5 Gen [4], and why-why analysis. It was found that the issue of the first complain was mainly caused by the process of gathering user requirement and testing in system development process (shows in dot square in figure 1). The process of gathering user requirement was performed by conducting focus group interview which merely rely on employee's experience, resulting in providing imprecise information and the system itself was not  Figure 1: The process of document system development perfectly function. In system testing, user acceptance test is currently used in which the financial institution has no standard procedure in determining test scenario. This may not literally cover all working functions so that the system error may reach the customer. The issue of the second and third complaints have been found that both issues caused by human error of the staff from the operation of improving the system but improper set the function back to the normal condition before operate the system. Unfortunately, the financial institution has no policy to check these documents before sending out to customer. Thus, wrong documents were sent out. Collecting requirements are acquiring and collecting process to ensure stakeholder's expectations and employ process modeling. Due to the vagueness of formulated requirements, many system development projects were fail due to poor collecting requirement process in the early process. The successful software system is based on required functionalities under particular limitations required from the system [5,6]. Mattmann et al. [6]. Also pointed out the need of systematically work during requirements collection and provided critical evaluation. Thus, the purpose of present study is to design framework for user requirement and usability testing in system development to prevent the future issue of improper functions in system development. This study also provides quality check policy for current document system to prevent any error reach to the customer which may cause by human error in work operation.

User Requirement and Usability Testing Framework
The user requirement and usability testing framework is proposed based on literature review regarding to current waterfall model [7,8], as seen in figure 1. The details of framework is shown in table 1. This framework has divided the process of gathering user requirement into 5 working steps according to Metacognition [9][10][11][12][13] and work process tool for systematic collecting user requirement. Each step was clearly identified the responsible persons, activity, input, and output. The first step is to specify the business need in order to set goal of developing system and team member. The second step is to assess the current working condition in order to form the current work process, work constraints, and relevant parties with the system. The third step is to identify problems/ inefficient workflow with current work process in order to propose solution and set "to-be" work process. The fourth step is to study the feasibility of the "to be" process, modified, and set system requirement. The final fifth step is to summarize system requirement in standard form (REQ-1, REQ-2) which were designed according to software requirement specification (SRS) [14]. This documents will be used for communication and documentation. The usability testing Framework has 4 working steps which applied from usability testing process [15,16], as shown in table 2. The first step is to set the test goal refer to the requirement documents from user requirement framework (REQ-1, REQ-2). The second step is to set the test scenario according to the requirement of the system in REQ-2, the team has to identify all scenario of using and all type of input to the system and output from the system. The third step is to identify the users and number of users for testing the system. The fourth step is to identify the test metrics such as success rate, error rate, efficiency in term of time on task, and user satisfaction etc. These metrics is selected based on the goal and objective of system testing. The result from usability could be used to revise the system and help detect some error in system functionality.

Quality check policy
Quality check policy is proposed regarding to the human error from the third-party company who responsible for improving and updating system that caused error in system issuing the documents to customer such as wrong period and form. The quality checking policy is to check the documents after Current work Summary of current work procedures (As-Is) and list of relevant parties 3. Identifying problems -Project manager -Section Manager who wants to develop -Staff performing and who is involved.

Study current problems and solutions.
Current problems (e.g. Customer complaints) Current work procedures (As-Is) A solution to the current and longterm solutions.

Specify the solution along with the work process.
A solution to the current and longterm solutions. Current work procedures (As-Is) Work procedures that need to be modified (To-Be)

Developing requirements
-Project manager -Section Manager who wants to develop -Third-party company 5. Study the feasibility of the process required (To-Be).
Work procedures that need to be modified (To-Be) Summary of possible system requirements

Setting requirements
-Project manager -Section Manager who wants to develop 6. Summary requirement Summary of possible system requirements Document report of requirement (REQ-2) it releases from system, including period checking, working function, form checking, balancing, date of financial transactions. The checking frequency is only one sample for each of 6 type of documents for checking the condition of function whether it is in proper working function. Beside from manual checking policy, the system was also added self-check function for the balancing by having the system take the initial balance plus with total deposits and subtracting total withdrawals, must equal the final transaction balance. If not, the system will cancel working flow automatically and report those who control the system.

Validation
The user requirement and usability testing framework is validated by using system usability scales (SUS) comparing between current work process and the new framework with 30 employees from the financial institute who have experienced in the process of developing document system with third party company. The SUS is 10 questions on 5 points-Likert scale from (1) totally disagree to (5) totally agree. The score ranges from 0 to 40, the higher the better. For quality check policy, the validation is conducted by piloting the quality check policy for 6 months and record all the complaints and error regarding the documents issuing from the system.

Results
The result from SUS revealed that employees rated the proposed user requirement framework as significantly higher SUS scores than their current work process of user requirement, as shown in table 3. As well as, the result of the proposed usability testing framework as significantly higher in SUS scores than the current user acceptance testing. After piloting the quality check policy for 6 months (January to June), the results of error detected before reaching to customer is shown in table 4. During these 6 months, the financial institute received no complaint from customer regarding wrong balance, wrong issue of document date, and its form. This showed that the quality check policy could help the financial institute prevent the quality issues from their documents system to customer.

Discussion
The designed framework would help the financial institute to shape their process for system development to be more precisely developed toward the need of the system, as well as other industries that may have the same issue. In terms of the framework, it literally helped the institution the better work procedures of documents' requirement and system testing. Additionally, the framework of developing information system in requirement and testing system will assist the institution work and collect documents systematically and detailly.

Conclusion
The purpose of present study is to design framework for user requirement and usability testing in system development to prevent the future failures of improper functions in system development and provides quality check policy for current document system to prevent any error reach to the customer which may cause by human error in work operation. The results revealed that employees had positive attitude toward the new framework and believed that it is better in term of usability of work process. For limitations, as the research is merely emphasizing on the system of issuing documents and studying only in the financial institution, as such, there would possibly be other factors that should be concerned about other requirements and testing system, which could be needed to conduct more research. However, there is inevitable to improve the work process quality to be more effective and efficient and reduce repetitive manual tasks.