{'fraction_non_alphanumeric': 0.024706943192064925, 'fraction_numerical': 0.009197475202885483, 'mean_word_length': 5.37047673750718, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
New criteria for proficiency assessment of clinical laboratories by interlaboratory comparisons

The participation of clinical laboratories in proficiency testing programs is an essential tool for quality management. Using a database provided by an accredited proficiency testing provider, this paper compares the performance of criteria in the assessment of proficiency of participant laboratories for measuring glucose. The analysis considered not only commonly used approaches but also newly proposed performance scores that are independent of the consensus effect, based solely on parameters associated with the assigned value defined by the reference material. The observed discrepancies in the performance rating of laboratories, according to the evaluation criteria employed, allowed the evidencing of inadequacies in the classifications based on scores using parameters affected by the consensus among participants. The present work allowed a better understanding about the impact of criteria applied in proficiency testing programs on the adequacy of evaluation of laboratories’ performance, thus contributing to ensure the reliability of clinical diagnostics and therapeutic procedures.


Introduction
Along the twentieth century, the increasing role of the clinical laboratory in medical decision making has led to a growing demand for a careful assessment of the reliability of laboratory results [1]. Thus, the accuracy and comparability of measurement results provided by laboratories are essential to guarantee the reliability of diagnoses and treatments [2][3][4]. Laboratories use a wide variety of techniques to quantify a large number of parameters, so the results may not be comparable across labs [3]. Independent quality assessment for accreditation of medical laboratories, guided by particular requirements for competence and quality described in the international standard ISO 15189 [5], was developed to monitor the performance of individual laboratories. It included the need of participation in appropriate proficiency testing (PT) schemes, aiming at evaluating the measurement capability of participants by interlaboratory comparisons (ILC) operated according to ISO/IEC 17043 [6].
Among the methods described in ISO 13528 to obtain the assigned value of a proficiency test item [7], against which the participants' data are compared, the most commonly used is the consensus value derived from participants' results. The appropriateness of this method, however, is highly dependent upon the quality of the results provided by the participants [8]. The effect of outliers, subpopulations, and the substantial increase in the uncertainty of the assigned value determined by the participants' consensus, mainly when few laboratory results are available [9], can, potentially, lead to 2 misclassification of laboratories' performance. This risk indicates the importance of using independent criteria, which allow the harmonization of results.
Ideally, PT schemes should provide an estimate of the measurement uncertainty and a statement of the metrological traceability of the assigned value [6]. However, this approach's costs hinder its use, and the procedure conventionally employed is the use of participants' PT results to calculate the target value and the standard deviation for the proficiency assessment.
The present study compares the results for the PT performance of clinical laboratories for measuring three distinct concentrations of glucose in human serum, evaluated according to six different scoring approaches. Four of the studied performance indicators consist of the most often employed criteria, which incorporate elements based on the consensus value obtained from the outcome of participants. Other two statistical designs are proposed for evaluation, composed of parameters entirely independent of the participants' measurement results. The six analyzed performance evaluation approaches do not represent costs' differences for PT implementation.

Materials and Methods
The present analysis was performed using the measurement results reported by participating laboratories of a proficiency testing round for glucose determination in samples of lyophilized human serum with three different glucose concentrations (91.08 mg/dL, 301.96 mg/dL, 143.33 mg/dL). The data were provided from a PT round performed by an accredited PT provider. A wide range of analytical systems, consisting of a great variety of reagent/analyzer combinations, were employed by 1737 participating laboratories.
The statistical designs used for performance evaluation of the PT participants were organized into two groups, considering the influence or not of the reported results of laboratories. Group 1 consisted of four combinations of parameters based on recommended criteria described in [7], all of them with some influence from the consensus value (equations 1 to 4).
Where x is the result reported by a participant; X cons , the assigned value obtained from the interlaboratory consensus amongst participants; X ref , the assigned value derived from the reference value of the provided PT item; σ cons , standard deviation for proficiency assessment derived from the results reported by the participants in the same round; σ consFL , the standard deviation for proficiency, determined using a fixed limit (FL) of 13 % of the assigned value derived from the consensus (σ consFL = X cons *(FL/2)); and u Xref , the standard uncertainty of the reference value of the provided PT item.
In four of the scoring methods used in this study, the standard deviation for proficiency assessment was established on a fit for purpose value. This approach is strongly recommended in the Harmonized Protocol for the PT in Analytical Chemistry published by the IUPAC (International Union of Pure and Applied Chemistry), and the advantages are widely discussed in the literature [7,9,10].
Group 2 consisted of two proposed combinations of parameters entirely independent of the participants' measurement capability (equations 5 and 6). Where σ refFL is the standard deviation for proficiency, determined using a fixed limit (FL) of 13 % of the assigned value defined by reference value of the PT item (σ refFL =X ref *(FL/2)).
After calculation by each one of the six protocols considered, the participants' results were classified according to their score value outcome, being recognized as acceptable those with │z│ ≤ 2, regardless of the statistical method applied [6,7].
The reference values (X ref ) for each round item, and their standard uncertainties (u Xref ), were obtained from the data provided by the assessment of homogeneity of the reference material, corresponding to each of the three items of glucose concentrations in lyophilized human serum [7,9].
The assigned values derived from the consensus of participants' results in a proficiency test round were calculated using the procedures described in the Algorithm A presented in the Annex C.3 of ISO 13528 [7]. The one sample t-test and chi-square test for variance were used to identify differences of acceptance ranges and assigned values between the six performance evaluation procedures.

T-test indicated differences between X cons and X ref for PT items 1 and 3; and
Chi-Square Test showed discrepancies between σ cons and σ consFL for PT items 1 and 2, at a significance level of 0.05. Figure 1 presents acceptance ranges of reported values for concentration of glucose in human serum associated with item 1 of the PT round, defined according to the six score of the two approaches' groups (equations 1 to 6). Item 1 values for X cons (94.05 mg/dL) and X ref (91.08 mg/dL) are indicated as the assigned values according to the correspondent procedure. The laboratory results represented in Figure 1 include those reported by six ordinary (A to F) and three expert () laboratories. Although laboratories A, B and C were within the acceptable range for the scores whose assigned values were based on the reference value of item 1 (X ref ), the participants have been judged to have an unacceptable performance by the two scores with assigned value derived from the outcome of the PT participants (Z-score and Z-score FL ), which are false rejections. Likewise, inconsistencies were observed for the performance evaluation results of laboratories D, E and F between the scores based on X ref , to which their reported results were out of acceptance range, and the two approaches based on X cons , that considered measurement result reported as satisfactory. Since for PT item 2 an agreement between X ref and X cons was observed, the evaluation discrepancies resulted from the differences between σ cons and σ consFL , occurring for scores considering the recommended fixed limits when compared with those that do not apply FL values in the standard deviation for proficiency assessment.

Conclusions
This paper presented a comparative performance analysis of distinct scoring criteria employed for evaluation of clinical laboratories by Proficiency Testing schemes, for measuring three concentration items of glucose in human serum. Six different scoring approaches were employed, being two of these newly proposed statistical designs based on parameters entirely independent of the participants' measurement results. Discrepancies in evaluation rates were observed for PT items for which differences between the assigned value derived from the reference material used in the PT round (X ref ) and obtained by the consensus value from participants (X cons ). If assigned values were coincident, inconsistencies could be produced by differences in the standard deviation for proficiency assessment between scores using or not a recommended fixed limit among their equation parameters.
Thus, acceptable ranges are affected not only by differences between the reference value of the provided PT item and consensus value obtained from the outcome of participants, but also by 4 discrepancies between values of the assessment standard deviation comprising parameters based on a recommended fixed limit and those derived mainly from the participants' consensus.
Significant differences between parameters used in each performance score approach studied resulted in discrepancies between the acceptable ranges associated with each scoring method, which represents a potential risk of an inappropriate decision of clinical laboratories' acceptance. Considering the consensus may be biased due to the use of faulty methodology by participants, score inadequacy can be prevented by seeking to apply reliable references to set the assigned value, ideally traceable to SI, utterly independent of the results reported by the PT participating laboratories.

Figure 1.
Acceptance ranges for reported results to item 1 of glucose concentration, according to the six scoring methods associated with the equations (1) to (6). Results reported by six ordinary laboratories (A to F) and three experts () are shown on each adequacy range. Evaluation discrepancies are indicated as false approval (+) and false rejection ().