{'fraction_non_alphanumeric': 0.035016796674827766, 'fraction_numerical': 0.015544041450777202, 'mean_word_length': 5.006839945280438, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A multinomial quadrivariate D-vine copula mixed model for diagnostic studies meta-analysis accounting for non-evaluable subjects

Diagnostic test accuracy studies observe the result of a gold standard procedure that defines the presence or absence of a disease and the result of a diagnostic test. They typically report the number of true positives, false positives, true negatives and false negatives. However, diagnostic test outcomes can also be either non-evaluable positives or non-evaluable negatives. In meta-analysis of diagnostic test accuracy studies, the existence of non-evaluable subjects is an important issue that could potentially lead to biased estimates of diagnostic test accuracy. In this paper we propose a methodology for the meta-analysis of diagnostic tests where we additionally account for non-evaluable outcomes of the diagnostic test. We assume independent multinomial distributions for the true and non-evaluable positives, and, the true and non evaluable negatives, conditional on the latent sensitivity, specificity, probability of non-evaluable positives and probability of non-evaluable negatives in each study. For the random effects distribution of the latent proportions, we employ a drawable vine copula that can successively model the dependence in the joint tails. Our methodology is demonstrated with an extensive simulation study and illustrated by meta-analysing diagnostic accuracy studies of coronary computed tomography angiography for the detection of coronary artery disease.


Introduction
Diagnostic test accuracy studies observe the result of a gold standard procedure that defines the presence or absence of a disease and the result of a diagnostic test. They typically report the number of true positives (diseased subjects correctly diagnosed), false positives (non-diseased subjects incorrectly diagnosed as diseased), true negatives (non-diseased subjects correctly diagnosed as non-diseased) and false negatives (diseased subjects incorrectly diagnosed as non-diseased). However, diagnostic test outcomes can be non-evaluable (Begg et al., 1986). In meta-analysis of diagnostic test accuracy studies, the existence of non-evaluable subjects is an important issue that could potentially lead to biased estimates of index test accuracy (Ma et al., 2014;Nikoloulopoulos, 2018e;Schuetz et al., 2012). Schuetz et al. (2012) studied different ad-hoc approaches dealing with diagnostic test non-evaluable subjects, such as non-evaluable subjects are excluded from the study, non-evaluable positives (non-evaluable diseased subjects) are taken as true positives and non-evaluable negatives (non-evaluable non-diseased subjects) are taken as false positives, non-evaluable positives are taken as false negatives and non-evaluable negatives are taken as true negatives, and non-evaluable positives as false positives and non-evaluable negatives as false negatives. In all of these approaches Schuetz et al. (2012) used the the bivariate generalized mixed model (BGLMM) proposed by Chu and Cole (2006) and concluded that excluding the index test non-evaluable subjects leads to overestimation of sensitivity and specificity and recommended the conservative intent-to-diagnose approach by treating non-evaluable positives as false negatives and non-evaluable negatives as false positives. Ma et al. (2014) proposed a trivariate generalized mixed model (TGLMM) approach to treat the nonevaluable subjects as missing data to adjust for potential bias. The TGLMM has originaly proposed by Chu et al. (2009) to account for potential correlations among sensitivity, specificity and disease prevalence as many empirical studies have shown the assumption of independence between the sensitivity/specificity with disease prevalence for a dichotomous disease status is likely to be violated (Brenner and Gefeller, 1997;Leeflang et al., 2009Leeflang et al., , 2013. Ma et al. (2014) with extensive simulation studies have shown that the intentto-diagnose approach (Schuetz et al., 2012) under-estimate both sensitivity and specificity, while the TGLMM under the missing at random (MAR) assumption gives nearly unbiased estimates of sensitivity, specificity and prevalence. Nikoloulopoulos (2018e) extended the TGLMM by rather using a trivariate vine copula distribution with normal and beta margins for the distribution of the random effects. Simulation studies showed that the TGLMM approach over-estimate sensitivity and specificity when the univariate random effects are beta distributed. Under the MAR assumption, the vine copula mixed model gives nearly unbiased estimates of test accuracy indices and disease prevalence.
A recurrent theme underlying the methodologies in Ma et al. (2014) and Nikoloulopoulos (2018e) for analysis in the presence of missing data is the need to make assumptions that cannot be verified based on the observed data. Throughout their papers the assumption of MAR is adopted. Nevertheless, it is natural to be concerned about robustness or sensitivity of inferences to departures from the MAR assumption.
In this paper we treat the non-evaluable subjects as non-missing under a missing not at random vine copula framework. In fact, we extend the bivariate copula mixed model proposed by Nikoloulopoulos (2015) to the quadrivariate case. Note in passing that a special case of the bivariate copula mixed model is the BGLMM. The bivariate copula mixed model in Nikoloulopoulos (2015) assumes independent binomial distributions for the true positives and true negatives, conditional on the latent pair of sensitivity and specificity in each study. In the proposed methodology for the meta-analysis of diagnostic tests where we additionally account for nonevaluable outcomes of the diagnostic test, we will assume independent multinomial distributions for the true and non-evaluable positives, and, the true and non evaluable negatives, conditional on the latent sensitivity, specificity, probability of non-evaluable positives and probability of non-evaluable negatives in each study. For the random effects distribution, we employ a quadrivariate drawable vine (D-vine) copula. D-vine copulas are a special class of vine copulas (Bedford and Cooke, 2002;Joe, 1996) that have become important in many applications areas such as finance (e.g., Aas et al. 2009;Nikoloulopoulos et al. 2012) and biological sciences (e.g., Killiches and Czado 2018;Nikoloulopoulos 2018b), to just name a few, in order to deal with dependence in the joint tails.
The remainder of the paper proceeds as follows. Section 2 introduces the multinomial quadrivariate Dvine copula mixed model for meta-analysis of diagnostic studies accounting for non-evaluable results. Section 3 contains small-sample efficiency calculations to investigate the effect of misspecifying the random effects distribution on parameter estimators and standard errors. Section 4 illustrates our methodology with data from a published meta-analysis for diagnostic accuracy studies of coronary computed tomography angiography for the detection of coronary artery disease. We conclude with some discussion in Section 5.

The multinomial quadrivariate D-vine copula mixed model
In this section, we introduce the multinomial quadrivariate D-vine copula mixed model. In Subsections 2.1 and 2.2, a D-vine copula representation of the random effects distribution with normal and beta margins, respectively, is presented. We complete this section with details on maximum likelihood estimation.

The multinomial quadrivariate D-vine copula mixed model with normal margins
We first introduce the notation used in this paper. The focus is on two-level (within-study and between-studies) cluster data. The data are y ij , i = 1, ..., N, j = 1, . . . , 6, where j is an index for the within study measurements and i is an index for the individual studies. The data, for study i, can be summarized in a 2 × 3 table with the number of true positives (y i1 ), true negatives (y i2 ), non-evaluable positives (y i3 ), non-evaluable negatives (y i4 ), false negatives (y i5 ), and false positives (y i6 ) ; see Table 1.
The diseased subjects have three possible states: true positive, non evaluable positive, false negative. The multinomial observation is therefore the number of diseased subjects where the diagnostic test is in each of its states. Hence, we assume that the true positives Y i1 and the non-evaluable positives Y i3 are conditionally independent and multinomially distributed given where V 1 is the latent sensitivity and V 3 is the latent probability of non-evaluable positives.
In a similar manner the non-diseased subjects have also three possible states: true negative, non-evaluable negative, false positive. Hence we assume that the true negatives Y i2 and the non-evaluable negatives Y i4 are conditionally independent and multinomially distributed given where V 2 is the latent specificity and V 4 is the latent probability of non-evaluable negatives.

The multinomial D-vine copula mixed model with beta margins
Inspired by Wilson (2018), we choose to define the random effects distribution over Hence one the one hand we transform the latent probabilities to ones with no unit sum constraints and on the other hand the latent sensitivity and specificity remain on the original scale since V j = X j for j = 1, 2.
The within-study model takes the form The stochastic representation of the between studies model is where C(·; θ) is a D-vine copula with dependence parameter vector θ and F (·; π, γ) is the cdf of the Beta(π, γ) distribution. The models in (4) and (5) together specify a vine copula mixed model with joint likelihood L(π 1 , π 2 , π 3 , π 4 , γ 1 , (3) where we use beta instead of normal marginal distributions. Below we transform the integral into an integral over a unit hypercube using the inversion method. Hence the joint likelihood becomes The copula parameter vector θ has the dependence parameters of the random effects model and they are separated from the univariate parameters (π j , γ j ), j = 1, . . . , 4. The parameters π 1 and π 2 are those of actual interest denoting the meta-analytic parameters for the sensitivity and specificity, while the parameters π 3 and π 4 denote the probabilities of non-evaluable positives and negatives, respectively. The univariate parameters γ 1 , γ 2 , γ 3 , γ 4 denote the variability of the random effects. In contrast with the model in the preceding subsection the meta-analytic parameters for the sensitivity and specificity are on the original scale and the copula parameter θ 12 of the random effects distribution represents how the latent sensitivity and specificity are associated.

Maximum likelihood estimation and computational details
Estimation of the model parameters can be approached by the standard maximum likelihood (ML) method, by maximizing the logarithm of the joint likelihood. The estimated parameters can be obtained by using a quasi-Newton (Nash, 1990) method applied to the logarithm of the joint likelihood. This numerical method requires only the objective function, i.e., the logarithm of the joint likelihood, while the gradients are computed numerically and the Hessian matrix of the second order derivatives is updated in each iteration. The standard errors (SE) of the ML estimates can be also obtained via the gradients and the Hessian computed numerically during the maximization process.
For the multinomial quadrivariate D-vine copula mixed model numerical evaluation of the joint pmf can be achieved with the following steps: 1. Calculate Gauss-Legendre (Stroud and Secrest, 1966) quadrature points {u q : q = 1, . . . , n q } and weights {w q : q = 1, . . . , n q } in terms of standard uniform.

Small-sample efficiency-misspecification of the random effects distribution
An extensive simulation study is conducted (a) to gauge the small-sample efficiency of the ML method, and (b) to investigate in detail the misspecification of the parametric margin or bivariate pair-copulas of the random effects distribution. We use pair-copulas that have different strengths of tail behaviour, namely the bivariate normal (BVN) copula (intermediate tail dependence), the Frank copula (tail independence), the Clayton copula (lower tail dependence), the Clayton copula rotated by 180 • (upper tail dependence), the Clayton copula rotated by 90 • (upper-lower tail dependence), and the Clayton copula rotated by 270 • (lower-upper tail dependence); for more details and their functional form see Nikoloulopoulos (2015Nikoloulopoulos ( , 2017Nikoloulopoulos ( , 2018b. We set the sample size and the true univariate and dependence parameters to mimic the data analyzed in Section 4. To make it easier to compare strengths of dependence, we convert the estimated parameters to Kendall's τ 's in (−1, 1) via the relations τ = 2 π arcsin(θ), Hult and Lindskog (2002), Genest (1987), and Genest and MacKay (1986), respectively.
• The ML estimates of the meta-analytic parameters are slightly underestimated under copula misspecification.
• The SDs are rather robust to the copula misspecification.
• The meta-analytic ML estimates are not robust to the margin misspecification.

Meta-analysis of coronary computed tomography angiography studies
We illustrate the multinomial quadrivariate D-vine copula mixed model for the meta-analysis of diagnostic accuracy studies accounting for non-evaluable subjects by analysing the data on 30 studies from a systematic review for diagnostic accuracy studies of coronary computed tomography angiography for the detection of coronary artery disease (Menke and Kowalski, 2016).
We fit the multinomial quadrivariate D-vine copula mixed model for both beta and normal margins and different pair copulas at the level 1; for levels 2 and 3 we use BVN copulas. In cases when fitting the multinomial quadrivariate D-vine copula mixed model, the resultant estimate of one of the conditional dependence parameters was close to the right boundary of its parameter space (that is clear indication that the model with a full structure provides more dependence structure that it is actually required; see Nikoloulopoulos, 2017), we used a truncated model, i.e., we captured the strongest dependence in the first tree and then just used the independence copulas in lower order trees, i.e. conditional independence. Joe et al. (2010) show that in order for a vine copula to have (tail) dependence for all bivariate margins, it is only necessary for the bivariate copulas in level 1 to have (tail) dependence and it is not necessary for the conditional bivariate copulas in levels 2 and 3, to have tail dependence. Hence one can either use BVN or independence copulas at levels 2 and 3 without sacrificing the tail dependence of the vine copula distribution.
Since the number of parameters is not the same between the models, we use the AIC, that is −2×loglikelihood +2× (#model parameters) as a rough diagnostic measure for goodness of fit between the models. The AICs showed that a (truncated) multinomial quadrivariate D-vine copula mixed model with Clayton copulas rotated by 180 • for both the C 12 (; τ 12 ) and C 34 (; τ 34 ) pair-copulas and the Clayton copula rotated by 90 • for the C 23 (; τ 23 ) pair-copula and beta margins (Table 4) provides the best fit.
Though typically the focus of meta-analysis has been to derive the summary-effect estimates, there is increasing interest in drawing predictive inference. Summary receiver operating characteristic curves (SROC) can be deduced from the D-vine copula mixed model with the sensitivity and specificity on the original scale through the quantile regression techniques developed by Nikoloulopoulos (2015). SROC essentially shows the effect of different model (random effect distribution) assumptions, since it is an inference that depends on the joint distribution. The model parameters (including dependence parameters), the choice of the pair-copulas, and the choice of the margin affect the shape of the SROC curve. Figure 1 demonstrates the SROC curve and summary operating points (a pair of average sensitivity and specificity) with a confidence and a predictive region from the best fitted multinomial quadrivariate D-vine copula mixed model.  Cln{ω • 1 , ω • 2 }: The C12(·; τ12), C34(·; τ34) and C23(·; τ23) pair copulas are Clayton rotated by ω1 and ω2 degrees, respectively; † : Best fit.

Discussion
We have proposed a multinomial quadrivariate D-vine copula mixed model for meta-analysis of diagnostic test accuracy studies accounting for non-evaluable subjects. Our general statistical model allows for selection of pair-copulas independently among a variety of parametric copula families, i.e. there are no constraints in the choices of bivariate parametric families of copulas and can also operate on the original scale of sensitivity and specificity.
For the random effects, we have used a quadrivariate D-vine copula distribution or a truncated at level 1 quadrivariate D-vine copula (conditional independence), which allows flexible (tail) dependence (Joe et al., 2010). We have proposed a numerically stable ML estimation technique based on Gauss-Legendre quadrature; the crucial step is to convert from independent to dependent quadrature points that follow a quadrivariate D-vine distribution.
Software R functions to implement the multinomial quadrivariate D-vine copula mixed model for meta-analysis of diagnostic tests with non-evaluable subjects will be part of the next major release of the R package CopulaREMADA (Nikoloulopoulos, 2018a).