{'fraction_non_alphanumeric': 0.023541197094916103, 'fraction_numerical': 0.013711495116453793, 'mean_word_length': 5.066464109380934, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 2, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A Revamp of the Internal Quality Auditing Process

In this paper, the researchers have described the development program used for the internal quality auditors of Company A. This program was developed to increase the competency of the auditors so that they could effectively conduct both internal and external audits. The competency of the auditors was an important requirement according to the various management standards like the ISO 14001:2015, ISO9001:2015, IATF 16949:2016, ISO45001:2018 and IECQ QC 0800000. Furthermore, this program exposed all the auditors to the internal audits according to the requirements of the ISO 9001 standards and the in-process quality audits, which were divided into 6 areas for 8 months. For ensuring the success of the program, 15% of the auditors’ Key Responsibility Areas (KRA) were attributed to their performance and their contribution to the general internal auditing program. The different Subject Matter Experts (SME) trained the auditors based on the requirements of the managing standards and their auditing skills. A monitoring technique was established for assessing the competency level of the auditors. The auditors, who showed a good performance in these programs, were recommended to partake in the Lead Assessors program and were asked to audit local or internalexternal suppliers.


Introduction
Internal audit falls on clause 9.2 of various management standards such ISO 9001, ISO 14001, ISO 45001 and ISO 27001, to name a few. The extent of internal audit programme is based on size , nature and complexity of a business. According to ISO 9001 standard [1], a quality audit included a 3-step process, i.e., determination of the auditability of all planned processes with regards to the objectives of the study; conformity of the activities with the objectives; and the efficiency of the activities in fulfilling the objectives [2]. The internal audit must be effectively conducted for determining the operational performance of the organisation [3]. Implementation of an audit programme needs to be monitored, measured and reviewed. Mustika [4] proposed the research model based on the factors which affected the effectiveness of the internal audit process, as shown in Figure 1.

Competence (+)
Relationship between Internal and External Auditors (+) Auditee Support (+) Internal Audit Effectiveness Figure 1. Factors affecting the effectiveness of the internal audit process [4] To have a successful internal audit programme, an organization needs to have competent auditors. To ensure competency, this pool of auditors is to be evaluated regularly. The researchers studied the developmental program for internal auditors in Company A which was ISO9001 certified. In this manufacturing sector, internal auditors did not solely carry out full-time audits. Hence, performing internal audit was not a priority to the auditors. A revamp on the internal auditing program was established to have competent auditors. This was set as one of company objectives. In order to achieve this, an auditor competency program was set up.

Literature Review
The internal audit programs determine if the organisations were 'fit for their stated purposes' [2]. An earlier study Asiedu and Deffor [5] mentioned that effective internal audit sessions could detect corruption. On the other hand, Naheem [6] stated that an effective internal audit process detected money laundering processes. Kabuye et al. [7] noted that internal audits detected fraud management. The internal audits review and monitor the operations carried out by an organisation and could suggest some steps for its improvement [8]. The use of novel technologies, processes and policies could help the organisations to ensure that the objectives of the company were fulfilled [9]. The factors which could affect the efficiency of the internal audit process included the competency, objectivity and independence of the auditors, audit reporting and planning [3,[10][11]. Roussy and Brivot [12] stated that an effective internal audit process required better management support. Tackie et al. [13] observed that these factors did not affect the effectiveness of the internal audit process in the organisation. Similar results were also noted earlier [13][14].
Based on the earlier results Sari and Susanto [15] it was concluded that the competency of the auditors significantly affected the audit quality and the supply chain for the information systems. Work experience was required for detecting the presence of any irregularities in the system. Results noted by Shamki and Alhajri [14] showed that it could significantly affect the auditors' performance with regards to the areas they were familiar with. Furthermore, Christ et al. [16] presented a rotational program set up for the auditors which acted as a training base for improving their work experience. Competent internal auditors carried out an effective auditing process. Some of the other characteristics of effective auditors were -skilful when talking to people, the ability to establish relationships; asking questions in an intelligent manner [17]. Lee and Park [18] stated that a larger number of internal auditors in the organisation contributed to more effective audits.
Rogala [19] observed that some of the auditors failed because of unsatisfactory training levels, lesser time for preparing audits (which required combining the activities of an internal auditor and other primary activities), and finally, a lack of recognition by the organisation's management for the auditor. Nwannebuike et al. mentioned that it was better to train the internal auditors instead of developing some error-proof internal auditing processes [20]. The recruitment process used for internal auditors must be clear and transparent. The organisation's management must always motivate internal auditors and cooperate with them during the auditing process. They also need to be trained regularly for improving their auditing skills. Similar results were noted earlier by [11].

Proposed framework
The auditing scope was initiated based on the ISO9001 standard. Evaluation criteria for competency of auditors was set up by a group of Subject Matter Expert (SME). Round 1 of the internal audit programme was done in 8 months for 6 different areas. One of the evaluation criteria for internal auditors was applying knowledge in specific disciplines in in-process areas. The in-process audits were categorised into six categories which were SERI (Special Engineering Request Instruction), TEI (Temporary Engineering Instruction), Qualification & Recipe (Tool Qualification & Recipe), SPC (Statistical Process Control), QRB (Qualification Review Board) and Corrective and Preventive Action System (CAPA).
The researchers selected 19 auditors from the technician and engineering levels. This pool of auditors are from different areas such as outgoing quality, in-process, quality management system, in-coming, statistical process control technicians and engineers. Auditors were initially internally trained based on the ISO9001 standard, in-process categories above and "generic" auditing processes and theories. Auditing process requires detailed planning skills. Audit dates, audit checklist and previous audit report follow up need to be prepared ahead. During audit, listening and asking questions techniques are needed to ensure audit objectives are met. After the audit, report writing skills and follow ups on action items or findings raised have to be completed to close the whole audit process.
Based on the internal audit areas scheduled, the auditor developed and submitted checklist to the SMEs for further reviewing. The SMEs reviewed this checklist and returned it for further editing or correction if needed. The SMEs then observed the complete auditing process and noted the auditing skills and performance of the auditors. After auditing process was completed, SME would review observations and comments to the auditor for future improvements. Auditor then submit audit report to SME for review before report was sent to the auditee for approval.
The SMEs would list the details regarding the audit that must be conducted. For each auditing subject, the SMEs listed the scope and the criticality factors necessary for preparing the checklists and the items which have to be verified, as described in Table 1.  Table 2 presents the guidelines used by the SMEs to grade the auditors with regards to every auditing procedure. 10 points were allocated for every category. A maximal of 40 points could be attained for every audit. Well prepared and on submitted ahead/onetime without much assistance required 5 to 7

SMEs' Audit Score Guideline
Prepared and submitted on time with certain level of assistance required. 1 to 4 Not fully prepared, delay in report submission and heavy involvement of supports required. Audit Process 8 to 10 Covered planned audited area within time, able to grasp overall system to access effectiveness 5 to 7 Focus on compliance evidences 1 to 4 Need SME intervention

Progression of the Auditors' Skill Level
The scores of the auditors were categorised based on their skill levels in Table 3. The SMEs were on default at Level 5 based on their auditing skill level. The progress of the auditor's process was tracked each month for determining if their skill improved. Understand to put requirements into audit questionnaires 2. Audit skills: a. Adequate skill to ask question b.
Adequate skill to look for compliance evidence c.
Basis skill of audit report updates d.
Audit capabilities limited to IQA, verification compliance to requirements e.
Audit capabilities limited to specific compliance audit, or simple element of ISO standard requirements 3. Supervision required: a. lose supervisor from mentor and requires input updates to ensure audit performed and audit report done correctly. b. Necessary coaching needed during actual audit, in terms of putting proper verbal question to ask, and looking for evidences.

2
1. Audit material know-how: x Adequate understanding audit requirements x Adequate understanding of linking specific requirements to ISO requirements to specific procedures x Able to structure audit checklist requirements 2. Audit skills: x Able to audit looking with equal skills to look for effectiveness of the system. x Able to write the audit reports correctly per audit report guidelines x Audit exposure for internal ISO audit x Audit capabilities limited on internal audit activities 3. Supervision required: x Certain level of supervision required to ensure checklist is adequate, coaching during audit and modification required on audit report 3 1. Audit material know-how: x Capable cover both Level 1 and Level 2 2. Audit skills: x Capable to make judgement when to review for effectiveness of the system x Able to make logistical arrangement of the audit with certain supervision required x Audit capabilities cover up to external audit (local audits) 3. Supervision required: x Minimum supervision required to facilitate smoothness of the auditor's audit preparation x Certain level of supervision required to perform external local audits

R. Rahmat et al. / A Revamp of the Internal Quality Auditing Process 682
1. Audit material know-how: x Capable cover all Level 1, Level 2, Level 3 2. Audit skill: x Capable cover all Level 1, Level 2, Level 3 x Capable to cover external audit (oversea) x Able to show the capabilities to work independently 3. Supervision required: x Limited supervision required 5 1. Exceed Level 4 with additional capabilities: x Understand to look on known ISO element in the bigger scope and link to the requirements in particulars for external audits x Able to look for effectiveness of the system and using known ISO elements as basis reference points x Able to provide coaching to other auditors 2. Able to be a SME to audit activities

Weightage for the KRA(Key Reseponsibility Area)
Before initiating the audit revamp program, the structure of the program was presented to the top management for their approval. It was agreed that 15% KRA was allocated to every auditor for their annual performance appraisal. Every auditor was assigned a minimal number of audits which they had to complete every year. For SME, the management established that 20% of the KRA was to be allocated for the project. The additional allocation was set aside for every SME as they had to ensure that the objectives of the process were fulfilled by the year-end. Audit results, reports and the nonconformance reports were to be further monitored so that they could be improved. The auditor levels were identified and reported. The auditors who scored >30 points were recommended to undergo the lead assessor program. Thereafter, they were allowed to conduct external auditing process (which supported the incoming quality group with regards to the suppliers auditing) in the international and local organisations.

Rotation Schedule
Each auditor was required to conduct audit on the in-process areas specified on rotation basis monthly. Figure 2 highlights the rotation schedule for the auditors. The detailed audit item to be conducted will be prepared by the relevant SME and communicated to the auditors at the first week of the month.   The auditors who conducted fewer audits were technicians who worked on shifts. It was observed that technicians received lower scores than the engineers. Table 4 presents an example of the data that was collected with regards to the auditor performance based on their areas. A few audits could not be completed since there were no samples or topics that could be audited, as determined by the SMEs. The status regarding the auditors' skill and training level after Year 3 of the program was described in Table 5. The number of auditors increased by >50%. After Year 3, 4 auditors were upgraded from Skill Level 1 to Skill Level 4.  Table 6 presents the number of audits which were conducted (external and internal) for 3 years. The auditing activities were seen to increase in Year 2 and decreased in Year 3 because of the business environment. In Year 2, the number of internal and external audits increased. The internal auditors with Skill 4 were selected for carrying out supplier audits, locally and internationally. This acted as a reward for auditors, who worked to upgrade their auditing skills.

Moving forward
Due to business demands, organizations now carry more than one management standard such as ISO14001 (Environmental Management System), ISO 45001 (Occupational Health and Safety Management System) and ISO 27001 (Information Security Management System). The internal auditing program can be further improved to have competent auditors who can conduct multiple management system audits in one single visit. Furthermore, these management systems are now aligned to one common structure. It is easier to train auditors on interpreting and auditing them. Current move now is to train auditors to conduct integrated audits. Audit resources will be better managed with this.

Conclusions and Future works
Auditors need to have good auditing skills and management standard knowledge to achieve competency. These skills include understanding the audit objective, planning, writing, listening, asking questions and following up on the findings raised during the audit. The audit revamp program set up included training and monitoring programs for auditors (engineers and technicians) from different work backgrounds. Different inprocess areas were audited on rotation basis. Subject Matter Experts who monitored the skill levels of the auditors and the standardised auditing score guidelines were part of the evaluation methods used. With practise, auditors will gain experience to perform better audits. Moving forward, organizations need to have versatile auditors who can conduct integrated audits to better manage resources. This will definitely be a bonus to the organisation.