{'fraction_non_alphanumeric': 0.019984118581259924, 'fraction_numerical': 0.006418740074113288, 'mean_word_length': 5.098870056497175, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Assessing the client’s risk management performance in construction procurement and contracting: Case studies

Today we are facing an increasingly uncertain world that can pose a threat to any business, but also an instance to leverage and generate new opportunities. Risk Management (RM) understands that present decisions and actions can influence the future, by identifying opportunities and threats. This paper addresses these issues with regards to the procurement and contract management of construction services in the mining industry, considering that this industry has not yet established a systematic framework for the application of RM in these functions. This situation results in lack of knowledge about RM performance and gaps in procurement and contracting, making an evaluation and monitoring system of this function a needed methodology. With such a system, gaps in RM can be identified, improvement actions recommended and performance controlled. Then, a prototype evaluation system has been constructed based on maturity models where the evaluation is based on: 1) a knowledge-based system that is able to propose actions to overcome current limitations of the RM function, and 2) a monitoring approach that is able to show the evolution of the RM function with the goal of improving it continuously. The maturity model and the prototype system used for the evaluation are described and results from two case studies corresponding to two mining companies are discussed. Also, main conclusions obtained from their analysis are presented. The prototype system has been validated through these cases and it might become the basis for the construction of a more functional system that could improve RM in procurement and contracting management in the future. The prototype has been considered as a useful tool by mining companies because it establishes a framework for RM in the management of procurement and contracting processes, can generate a systematic evaluation of RM in this area, and is of easy access and use.


Introduction
Nowadays there exists a global tendency to externalize the procurement of goods and services, and the mining industry is not the exception. In studies and interviews with Chilean mining companies, these companies estimate that the external procurement of goods and services involves around 70% to 90% of the total annual operational expenses of the company. It is important to consider that a report by the Chilean Mining Council (Consejo Minero) that integrates the 17 largest Chilean mining companies indicates that the operational annual expense of these companies in 2010 reached the amount of US$ 15,110 millions [1].
At the same time, the majority of these companies have not been able to adjust their processes and systems appropriately to deal with the challenges associated to this situation, like: 1) an increase of the number of contracts, their complexity, diversity and interdependency; 2) a high competitive pressure and the need to maintain long-term stable relationships with suppliers and collaborators; 3) the involvement of a very high quantity of resources, both human and monetary; and 4) the increase of the uncertainty around suppliers' performance, which represents a real threat for the mining business.
In front of this scenario, the own participants of Chilean mining companies state that there is not a systematic vision of risk management (RM) inside the companies that help to manage uncertainty in the area of procurement and less of a formal reference to support risk management in this important function. As stated by Hillson [2] organizations that wish to implement a formal approach to RM or to improve the existing one, need a framework against which to contrast their current practices. This paper presents a research aimed to contribute to create a framework for implementing or improving risk management practices in procurement activities in mining companies by proposing a knowledge-based system. The research developed a computer prototype that was applied to two mining companies that were used as case studies. In the next sections a brief background of the research is presented as well as the definition of the research problem and methodology. Also, the characteristics of the knowledge-based system are discussed and the computer prototype is described. Finally, the main conclusions of the research are presented.

Background
This section presents a discussion of the main fields of knowledge that were applied to develop the risk management framework: risk management maturity models and knowledge-based systems.

Risk management maturity models
A maturity model is a form of evaluation that provides a systematic framework to conduct benchmarking and performance improvement in a particular area [3], leading the organization strategically and linking to continuous improvement [4]. It has been shown that organizations improve their level of project management maturity experience lower costs, increase the predictability of their schedules and improve quality. Also, there is a positive relationship between high levels of maturity in project management and an increased performance of their projects [5].
The development of a RM maturity model for procurement management in mining companies is relevant, because the development of a complete risk management process tends to be more useful when projects involve significant resources, are unique, have a long planning horizon, are large, are highly complex involving several organizations, or may be affected by political issues [6] as is the case with most of mining projects.
Several risk management maturity models have been proposed in the literature. Jia et al. [7] report that maturity models associated with risk management are proposed by [2] and [8]. Other models are proposed by [9] and [10]. The latter presents a different approach, describing in detail the characteristics of each maturity level according to the stages of risk management raised by the PMI [11].

Knowledge-based systems
They are computer programs that simulate the processes of problem solving of human experts in specific fields. In its simplest form, a knowledge-based system consists of a knowledge base and an inference engine [12]. A knowledge-based system offers several advantages such as to improve the accuracy of decisions in less time, and to retain and store tacit knowledge from experts/individuals with experience, allowing easy access and knowledge sharing [13,14].
The reasons for building a knowledge-based system usually have to do with the need for a solution of knowledge management i.e., an operation within the organization requires expertise that is not available often enough, or is not fully exercised. The most common problem is experience that is not sufficiently available or experts may simply be too busy to answer all queries that require their expertise. Alternatively, experts can be frequently be used in routine cases, not optimizing the use of their knowledge [15].
Knowledge is an important resource for risk management because there are many activities that need experience and knowledge to be resolved, like risks identification and risk responses, for example.

Research problem
In general, it has been observed that there are different proposals of generic models for managing risk, but none focused on the RM in the procurement management (PM). Additionally, interviews with managers of the PM in mining companies and in the review of the literature show that there is not a development of an evaluation system for risk management in this area, and even less, the developing of an automated platform for evaluation and monitoring.
With the above background and those raised in the theoretical framework, this research proposes a method for assessing the maturity level of procurement' risk management in an organization in order to propose actions for improvement, based on a computational prototype to evaluate and monitor the evolution of the key factors of RM in the PM. Thus, by using computational tools, we seek to create a system for evaluating and monitoring the maturity level of RM in PM in a simple and practical way by carrying to practice the identification and characterization of the key factors of this area of project management.

Research methodology
The research was conducted in four main phases as follows: (1) to define the maturity model of RM in the PM, (2) to develop a measuring instrument associated to the proposed model, (3) to design and develop a knowledge-based and computational prototype system, and (4) to carry out a case study in two mining companies.
The first phase was carried out through literature review, critical analysis of existing maturity models, and structuring of the proposed model, which was validated with contributions of RM experts. The second phase was realized starting from the results of the first phase and the measuring instrument consists in a questionnaire that was validated using a pilot study with professionals from one mining company and then calibrating the questionnaire using their feedback regarding the consistency between the answers and the current reality of the company in the application of RM in PM.
To design and develop the knowledge-based system the framework proposed by [14] was used as a reference, to which a stage of validation and verification system has been added. The process applied was as follows: (1) to establish the structure of the knowledge-based system, (2) to construct the knowledge-based system, with computer representation of the basic rules of knowledge, (3) to construct the prototype and to integrate the knowledge-based system, (4) to test the prototype in some organizations to validate and verify the system, and (5) to adjust the system and incorporating the obtained recommendations.
In phase four, case studies were applied. As proposed by [16], this method reflects the nature of the investigation, because the research question is how or why, contemporary events are addressed, and events under research cannot be controlled. In addition, the case study can be used when the objective is to make a thorough investigation of certain process preserving the overall vision of the phenomenon, and no intention to establish a generalization in the statistical sense exists [17], which are the own characteristics of this research. As a result of case studies, responses to the questionnaires allowed knowing the level of maturity of risk management in the PM, and results were statistically analyzed. In parallel, the internal consistency was measured by Cronbach's Alpha coefficient statistic [18].

Main research results
As reported by Howard and Serpell [19], this research initially provided the maturity model used for evaluating risk management in PM and its associate levels of evaluation with five categories. To find about the most appropriate characteristics that a maturity model should have, the maturity-evaluating questionnaire proposed previously by Wolbers [20] was applied to 68 contract administrators of an important Chilean mining company to test the questionnaire and its consistency.
After obtaining the answers, maturity results were shown to contract administrators who reacted informing that maturity results showed an over estimation of the organization's reality in most of the evaluated factors. Also, other limitations were mentioned by the same respondents regarding the questionnaire structure and the kind of questions (using a Likert-like scale of 7 levels, many open questions that were less responded by a majority of the contract administrators, a very long questionnaire that required a long time for answering it, the lack of control questions and a paper format that did not help with the process). All these issues were considered in the development of a new questionnaire.
This new questionnaire was applied in a pilot study to 12 professionals from different Chilean mining companies. With this feedback final adjustments were made and the questionnaire was calibrated correctly. Figure 1 shows the maturity model used as the base for the questionnaire as reported by [19] and included in the final system prototype.

The RM system prototype
The prototype of the knowledge-based system aims to become a support system for decision-making, by capturing data, processing it and delivering a diagnostic of the maturity of the risk management function in procurement management and proposing an action plan to increment the maturity continuously, considering that this function is a key process in the mining industry as was discussed above. Figure 2 shows the main components of the prototype. As shown in the figure, the main components of the system are: 1) a knowledge base that contains the necessary knowledge domains to solve problems codified in terms of rules; 2) the working memory that contains the data input by the user when answering the maturity questionnaire; and 3) the inference engine that uses the input, processes it and compares with the knowledge base, producing in this way the diagnostic. The procedure for application of the prototype follows the next steps: 1) The RM manager sends an invitation for participation to all persons related to procurement management; 2) The personnel answer the questionnaire through Internet and the total number is checked; 3) If the number is very low, then the invitation is repeated until enough people respond the questionnaire; 4) Answers are recovered, input, and processed by the prototype and used to obtain a diagnostic of the current maturity of RM in PM; 5) Check for the existing gaps and recommend actions to improve the identified gaps.
The figure 3 shows a screen of the prototype when informing the user about the results of the diagnostic of the risk management function in PM.

Figure 3 Prototype -Diagnostic section
Once the diagnostic is delivered, the system provides a new screen that is called Identification of Gaps that shows if some gaps exist or not and then provide recommendations for each dimension to improve the maturity level of the RM function in PM. Figure 4 displays this screen.

Case studies
The evaluation was applied to two mining companies. One is a company that extracts mineral once the other is a supplier of goods and services to them. In the first company, the six persons that work in the area answered the questionnaire while in the second company 15 out of 20 persons did. Table 1 shows a summary of the results of these applications. Organization's culture 3 3 Process structure 3 2 Experience and importance of RM in PM 3 2 Application of the RM process 3 2

Conclusions
The main conclusion is that it is possible to improve the risk management as applied in procurement management by helping companies to evaluate their RM status and then, to improve the existing situation by applying recommendations given by the system or by experts. A system like this makes RM transparent and offers an understanding of the main RM factors to be addressed by decision makers in this area.