{'fraction_non_alphanumeric': 0.04517305938265086, 'fraction_numerical': 0.015745763525514857, 'mean_word_length': 4.559647718174539, 'pattern_counts': {'":': 0, '<': 6, '<?xml version=': 0, '>': 2, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 7, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Precision for binary measurement methods and results under beta-binomial distributions

To handle typical problems from fields dealing with biological responses, this study develops a new statistical model and method for analysing the precision of binary measurement methods and results from collaborative studies. The model is based on beta-binomial distributions. In other words, we assume that the sensitivity of each laboratory obeys a beta distribution and the binary measurement results under a given sensitivity follow a binomial distribution. We propose the key precision indicators of repeatability and reproducibility for the model and derive their unbiased estimates. We further propose a confidence interval for repeatability by applying the Jeffreys interval, which utilizes the assumption of beta distributions for sensitivity. Moreover, we propose a statistical test for determining laboratory effects, using simultaneous confidence intervals based on the confidence interval of each laboratory's sensitivity. Finally, we apply the proposed method to real-world examples in the fields of food safety and chemical risk assessment and management.


Introduction
One of the aims in conducting collaborative studies; namely, studies that several laboratories measure the identical objects along with the same protocol and compare the obtained measured values; is to evaluate the precision of new measurement methods and results. International Organization for Standardization (ISO) 5725 Parts 1 [5] and 2 [6] are widely used to conduct such studies and analyse the obtained results. ISO 5725-2 [6] assumes the obtained measured values are produced from a population following normal distributions, and uses a one-way analysis of variance (ANOVA) to evaluate precision. By contrast, several ISO committees and industrial agencies are now interested in dealing with binary measurement methods and results. For example, ISO/ Technical Committee (TC) 34 (Food products) / Subcommittee (SC) 9 (Microbiology) discuss studies on the detection of Listeria monocytogenes in foods [14] and on new real-time polymerase chain reaction (PCR) assays for detecting transgenic rice [4], both of which provide binary measured values.
Several statistical methods to analyse binary measured values have recently been proposed ( [2,3,8,14]). Wilrich [14] modified the basic model in ISO 5725-2 and directly applied it to evaluate the accuracy of binary measurement methods and results. This model assumed the obtained binary measurement results were produced from a population obeying some binomial distributions. Because this assumption is very natural for handling binary data and the model is easy for users to understand, Wilrichs work is one of the mainstream methods used for binary measurement methods and results.
Wilrich [14] conducted statistical tests to detect laboratory effects by applying a test for independence in contingency tables, that is, applying a Chi-squared test. However, it is known that Chi-squared tests are applicable under two conditions, np ≥ 5 and n(1−p) ≥ 5, where n is the number of repetitions in each laboratory and p is a binomial probability. From these conditions, the number of repetitions n must be more than 10. Wilrich [14] mentioned this limitation, but proposed no alternative methods. However, many real-world examples of collaborative studies dealing with biological responses do not fulfill the requirement. Such collaborative studies are important in several fields, such as food safety, chemical risk assessments and managements, and so on; see examples in Section 4. Note that the Organisation for Economic Co-operation and Development (OECD) has a key role in discussing how to conduct chemical risk assessments and management internationally, and the results of collaborative studies provide important information for OECD test guidelines. However, there are no details on how to conduct statistical analyses in the related OECD guidance documents [10].
The aim of this study is to present new methods for analysing the precision of binary measurement methods and results. First, we provide a new model to evaluate the precision of binary measured values, based on beta-binomial distributions. Second, we propose a statistical test method that does not rely on any normal-approximation techniques. Our research is an extension of Wilrichs approach [14], but it aims to more appropriately analyse real-world examples discussed in some public and industrial agencies and to overcome the aforementioned limitation. Further, we demonstrate some real examples.
This paper is organized as follows. Section 2 prepares notation and summarizes basic relevant facts. Section 3 presents the main results. First, we introduce the basic model and estimates of precision. Then, we define a Jeffreys-type confidence interval and propose an approach to statistically testing whether laboratory effects exist. In Section 4, some real-world examples are analysed using our model. Section 5 summarizes the findings of the study.

Notation
In the present study, the following notation is used: • L: the number of laboratories participating in collaborative studies.
• n: the number of measured values, or repetitions, in each laboratory.
• y ij : a random variable that describes the measured value of trial j at laboratory i. In other words, y ij ∈ {0, 1}, where 0 and 1 respectively indicate negative and positive values.
• x i : a random variable that describes the measured value of trial j at laboratory i. In other words, x i = n j=1 y ij .
• p i : a random variable that describes the probability of measured values belonging to the positive category, that is, sensitivity, at laboratory i.
• p: the expectation of p i , that is, E(p i ).
• Be(q): a Bernoulli distribution with a probability density function: where 0 < q < 1.
• Bi (n, q): a binomial distribution with a probability density function: where m is a nonnegative integer and 0 < q < 1.
• Beta(a, b): a beta distribution with a probability density function: where a, b > 0 and B(a, b) is a beta function defined as: • BBi (m, a, b): a beta-binomial distribution, as explained in Section 2.4.
•X: an estimate of X, where X stands for any statistic.

Precision in ISO 5725
The ISO 5725 series (e.g., ISO 5725-1 [5]) defines the accuracy of measurement methods and results as general terms involved with trueness and precision. Trueness, defined as the closeness of agreement between the average value obtained from a large series of measured values and an accepted reference value, is usually expressed in terms of bias. In other words, it is the difference between the expectation of the measured values and the accepted reference value. Precision, defined as the closeness of agreement between independent measured values obtained under stipulated conditions, is usually expressed in terms of standard deviations of the measured values.
To clarify the precision of measurement methods, two measures are typically used: repeatability and reproducibility. Repeatability refers to measured values under repeatability conditions; namely, independent measured values obtained with the same method using identical test objects in the same laboratory by the same operator using the same equipment within short intervals of time. Reproducibility relates to reproducibility conditions; namely, measured values are obtained with the same method on identical test objects in different laboratories with different operators using different equipment.
In the ISO 5725 series, the basic model for measured values to estimate accuracy of a measurement method is as follows: where m, B, and e denote a general mean (expectation), a laboratory component of variation (under repeatability conditions), and a random error (under repeatability conditions), respectively. Moreover, the expectation of B is assumed to be 0 and its variation is called a between-laboratory variance, which is denoted by σ 2 L . The expectation of e is assumed to be 0 and its variance, called a within-laboratory variance, is assumed to be identical in all laboratories. Repeatability variances σ 2 r and reproducibility variances σ 2 R are, respectively, defined as follows:

ISO-based method for binary measurement methods and results
This subsection briefly summarizes an ISO 5725-based method, originally proposed by [14].
The basic model to analyse binary measured values is as follows: where y ij is the measured value defined as 0 (negative) or 1 (positive) for trial j ∈ {1, . . . , n} at laboratory i ∈ {1, . . . , L}; p i is sensitivity, the probability of obtaining a measured value y ij = 1 at laboratory i; and p is its expectation. This model is based on the basic model of ISO 5725-2 [6].
To estimate the repeatability, between-laboratory, and reproducibility variances, a one-way ANOVA (a random effects model) is performed. From Table 1, we have the following estimates:

Beta-binomial distribution
Definition 1. A random variable X follows a beta-binomial distribution if the probability density function of the variable X is defined as follows: where a, b > 0 are nonnegative real numbers, and B(a, b) is a beta function defined as: Remark 1. A beta-binomial distribution is a compound distribution assuming that a defective ratio parameter, or a binomial probability, p of a binomial distribution follows a beta distribution.
If a random variable X follows a beta-binomial distribution BBi(a, b), then the expectation and variance of X are, respectively, as follows:

Main results
This section first introduces a new model using a beta-binomial distribution, and then provides estimates of precision measures based on the model.

Our basic model
We propose the following basic model of the measured value y ij ∈ {0, 1} to evaluate the precision of binary measurement methods: where y ij is the measured value defined as 0 (negative) or 1 (positive) for trial j at laboratory i, and p i is a random variable that describes the probability of obtaining a measured value y ij = 1, or sensitivity, for laboratory i. In other words, if we let x i |p i = n j=1 y ij |p i and x i = n j=1 y ij , then x i |p i and x i are assumed to follow a binomial distribution Bi(n, p i ) and a beta-binomial distribution BBi(n, a, b), respectively.

Repeatability and reproducibility variances
First, we show the theoretical values of σ r , σ L , and σ R . Proposition 1. Assume y ij follows a beta-binomial distribution BBi(n, a, b). Then, the repeatability, between-laboratory, and reproducibility variances are, respectively, calculated as follows: , , A proof of Proposition 1 is shown in Appendix A.1. Since the variance of the beta-binomial distribution BBi(n, a, b) is nab(a + b + n)/((a + b) 2 (a + b + 1)), from Proposition 1 the proposition below holds: Proposition 2. Among variances σ r , σ L , σ R , and σ BBi , the following relations hold: where σ 2 BBi is a variance of a beta-binomial distribution. Furthermore, σ 2 R can also be expressed as A proof of Proposition 2 is shown in Appendix A.2. Using Proposition 2, we obtain estimates of sensitivity and the three variances.
Proposition 3. The following are unbiased estimates of p i , σ 2 r , σ 2 L , and σ 2 R , respectively: whereσ BBi is an estimate of a variance of a beta-binomial distribution, that is, A proof of Proposition 3 is shown in Appendix A.3.

Laboratory effects
A major objective of collaborative studies is to check whether laboratory effects exist. We propose to use simultaneous confidence intervals of sensitivity p i . Let each I i be α/L%-confidence interval of p i (0 < α < 100), and I := L i=0 I i . Then, if I = ∅, one can conclude that there exists laboratory effects with α%-significance level; otherwise, this cannot be concluded.
We define a Jeffreys-type confidence interval and proposes to use the interval for each confidence interval of sensitivity p i . Definition 2 (Jeffreys-type confidence interval). Let n and x be the numbers of measured values and of positive values, respectively. Let β(α; a, b) be the 100α% value of Beta(a, b), and letã = x − a + 1 andb = n − x − b + 1. Then, a Jeffreys-type 100(1 − α)% confidence interval of p, say CI J (p), is defined by: where The Jeffreys-type confidence interval is a modified interval of the Jeffreys interval; see [9] and [12]. The method was based on Bayesian estimate concepts. For any i ∈ {1, . . . , L}, p i follows a beta distribution Beta(a, b) and x i |p i follows a binomial distribution Bi(n, p i ). Then, from the Bayesian rule, we have: where C and C are normalization constants. This expression implies that p i |x i follows a binomial distribution Bi(x i − a + 1, n − x i − b + 1). Thus, we propose the expression (6) as a confidence interval of each p i . The values of parameters a and b are unknown in general. Therefore, estimates of a and b are used in real-world examples to calculate CI J . Since p i follows a beta distribution Beta(a, b), we havê p i = a/(a + b). Furthermore, from Proposition 1, we have a + b = σ 2 r /σ 2 L . Thus, we obtain: By replacing σ 2 r and σ 2 L with their estimates from Proposition 3, we propose that: are used to calculate the expression (6) in real examples. Finally, we emphasize that this confidence interval utilizes our assumption of beta-binomial distributions.

Numerical examples 4.1. h-CLAT
This subsection analyses the results of a collaborative study on the human cell line activation test (h-CLAT) [1]. The h-CLAT is an in vitro assay for evaluating the skin senstization potential of chemicals  1  3  2  3  3  1  4  3  5  3 without animal experiments, and is registered as the OECD test guideline 'Test No. 422E: In Vitro Skin Sensitisation' [11]. The OECD has a key role in the field of chemical risk assessments and managements, and the OECD test guideline test is considered to be highly accurate.
To evaluate the accuracy of h-CLAT, Sakaguchi et al. [13] conducted a collaborative study. It consisted of five laboratories and each laboratory repeated measurements three times, that is, (L, n) = (5, 3). Each laboratory measured 21 chemicals, but this paper focuses on two particular chemicals, hydroquinone and propyl gallate, as an example.
First, the results for the case of hydroquinone are shown in Table 2. In the table, the first and second columns show laboratory number and the number of detections of the skin senstization potential, respectively. From Proposition 3, we obtain the following estimates: Since p is the expectation of p i , we havep = 0.87. From (7) and the above results, we haveâ = 0.867 andb = 0.133. For i = 1, 2, 4, 5, the 99.0% Jeffreys-type confidence intervals of p i are calculated by the upper and the lower 0.5% values of Beta(3.13, 0.867) and for i = 3, they are calculated by Beta(1.13, 2.87). We note that we choose 99.0% confidence interval for each p i since the number of laboratories is five. Therefore, we obtain:  potential, respectively. From Proposition 3, we obtain the following estimates: In other words, the simultaneous interval is not empty. Thus, we cannot say that there exists laboratory effects with 95%-significance level.
Remark 2. Note that the collaborative study has only five laboratories with three repetitions, but such small collaborative studies are typical in the field of chemical risk assessments and managements.

Listeria monocytogenes
This subsection analyses the results of a collaborative study on Listeria monocytogenes, which was presented in ISO 16140 [7] and analysed by Wilrich [14]. The study involved ten laboratories and each laboratory repeated the measurements five times, that is, (L, n) = (10, 5). The results are shown in Table 4. In the table, the first, second, and last columns show laboratory number, the detected results on L. monocytogenes, and the number of detections, respectively. Here, y ij = 1 and 0 indicate respectively that L. monocytogenes was and was not detected, respectively. From Proposition 3, we obtain the following estimates: Since p is the expectation of p i , we havep = 0.92. Next, from (7) and the above results, we haveâ = 0.246 andb = 0.0214. For i = 1, 2, 3, 4, 6, 8, 9, 10, 99.5% Jeffreys-type confidence intervals of p i are calculated by the upper and the lower 0.025% values of Beta(10.8, 0.979) and for i = 5, 7, they are by Beta(6.75, 4.98). We note that we choose 99.5% confidence interval for each p i since the number of laboratories is ten. Therefore, we have:

Discussion and concluding remarks
This study introduced a new method for evaluating the precision of binary measurement results. The key idea was to assume beta-binomial distributions of the data in collaborative studies, in contrast to the binomial distributions assumed in a similar method proposed by Wilrich [14]. The indicators of repeatability and reproducibility of the present and previous studies were found to be identical when Wilrich's binomial probability p was replaced by the expectation E(p i ) of our p i . The present also study proposed how to construct confidence intervals for estimates of p i . Our confidence interval was applied using a Jeffreys interval based on beta distributions. Under the assumption that p i obeyed a beta distribution, the Jeffreys-type confidence interval was derived by applying percentage values of an estimated distribution of the beta distribution.
Furthermore, we present a statistical test method to detect laboratory effects, using the Jeffreystype confidence intervals of each p i and the simultaneous confidence interval. We applied the proposed test method to two real-world examples in chemical risk assessments and managements and in food safety. Since Wilrich [14] used a test for independence in contingency tables, that is, a Chi-squared test, for checking the homogeneity of binomial probability p, we could not conduct a statistical test on the example in Section 4.2 to detect laboratory effects because of the small number of repetitions. However, collaborative studies with small numbers of repetitions are sometimes conducted. The two collaborative studies analysed here were both real-world examples that provided important informations for discussions in public institutions, such as the Organisation for Economic Co-operation and Development (OECD) and International Organization for Standardization (ISO).
Finally, we note some further challenges. First, Proposition 3 proved that the proposed estimates of repeatability and reproducibility variances were unbiased. While this is a good property for estimates, unbiased estimates are generally not unique. If the efficiency of the unbiased estimates can be proven, then our proposed estimates are more appropriate; but proofs are not given. Second, we proposed a confidence interval and a statistical test method for detecting laboratory effects, but the confidence intervals had wide ranges. As a result, the simultaneous confidence intervals also had wide ranges. In general, such situations are typical in biological fields. Thus, we need to consider whether our proposed methods and results are sufficient for decision-making in such application fields. Third, the reproducibility variances strongly depended on only p i . This implies that, for any measurement methods, only sensitivity decides their reproducibility. In other words, our model cannot describe the difference between two measurement methods with the same sensitivity but different precisions. This problem is a well-known property of binomial distributions, and our results show that beta-binomial distributions cannot overcome it. These are considerations future work and applications.

A.1. Proof of Proposition 1
Proof. Since y ij |p i follows a Bernoulli distribution Be(p i ), the repeatability variance in laboratory i is σ 2 r(i) = p i (1 − p i ). Therefore, as p i follows a beta distribution Beta(a, b), the repeatability variance σ 2 r = E(σ 2 r(i) ) is as follows: Next, we consider the meaning of between-laboratory variance, σ 2 L = V (p i ). Since p i follows a beta distribution Beta(a, b), we have: Finally, from (8), (9), and the definition of reproducibility variance: σ 2 R = σ 2 r + σ 2 L , we have: ab (a + b)(a + b + 1) + ab (a + b) 2 (a + b + 1) = ab (a + b) 2 .

A.3. Proof of Proposition 3
Proof. It is well known that (2) is an unbiased estimate of sensitivity p i . From the definition of σ 2 R , (5) is an unbiased estimate if (3) and (4) are unbiased estimates. Therefore, this subsection only proves that (3) and (4) are unbiased estimates.
First, we prove that (3) is an unbiased estimate. Fromp i = (1/n) n i=1 y ij = x i /n, we have: Since x i follows a beta-binomial distribution BBi(n, a, b), we obtain: Next, we prove that (4) is an unbiased estimate. Sinceσ 2 r andσ 2 BBi are unbiased estimates, the following holds immediately: