{'fraction_non_alphanumeric': 0.04364354480375216, 'fraction_numerical': 0.02187114292767218, 'mean_word_length': 4.854335260115607, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
The screening value of RT-LAMP and RT-PCR in the diagnosis of COVID-19: systematic review and meta-analysis

The purpose of this systematic review is to evaluate the test accuracy of reverse-transcription loop-mediated isothermal amplification (RT-LAMP) and reverse transcription-PCR (RT-PCR) for the diagnosis of coronavirus disease 2019 (COVID-19). We comprehensively searched PUBMED, Web of Science, the Cochrane Library, the Chinese National Knowledge Infrastructure, and the Chinese Biomedical Literature Service System until September 1, 2021. We included clinical studies assessing the sensitivity and specificity of RT-PCR and RT-LAMP using respiratory samples. Thirty-three studies were included with 9360 suspected cases of SARS-CoV-2 infection. The RT-PCR or other comprehensive diagnostic method was defined as the reference method. The results showed that the overall pooled sensitivity of RT-PCR and RT-LAMP was 0.96 (95 % CI, 0.93−0.98) and 0.92 (95 % CI, 0.85−0.96), respectively. RT-PCR and RT-LAMP had a 0.06 (95 % CI, 0.04−0.08) and 0.12 (95 % CI, 0.06−0.16) false-negative rates (FNR), respectively. Moreover, subgroup analysis showed mixed sampling and multiple target gene diagnosis methods had better diagnostic value than single-site sampling and a single target gene. The sensitivity and FNR were also significantly affected by the reference method. Comparing RT-LAMP with established suboptimal RT-PCR may exaggerate the performance of RT-LAMP. RT-PCR and RT-LAMP showed high values in the diagnosis of COVID-19, but there was still a FNR of about 6%–12%.


Introduction
Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is the causative agent of the coronavirus disease 2019 (CoVID19) pandemic. The control and management of this pandemic have faced unprecedented challenges (Wiersinga et al., 2020). Isolating infected individuals, contact tracing, extensive diagnostic testing, and vaccinations are crucial to limit this infection (Bosetti et al., 2021). Accurate diagnosis is the basis to ensure the effective implementation of the above strategies.
Reverse transcription-PCR (RT-PCR) and reverse-transcription loopmediated isothermal amplification (RT-LAMP) are two widespread testing methods, and each has distinct advantages. Evidence to date indicates that RT-PCR is considered the most sensitive method for the detection and quantification of SARS− COV-2 nucleic acid (Walsh et al., 2020). RT-LAMP is a reliable and rapid screening test, which can be used in the field or under non-laboratory conditions (Ganguli, 2020).
However, there are analytical and interpretation issues with the molecular detection results of SARS− COV-2 infection. The false negative results of molecular tests would need detailed analysis (Bohn et al., 2020;Xiao et al., 2020).
Most clinical studies assessed agreement between different nucleicacid amplification tests (NAATs) to evaluate the diagnostic accuracy. Generally, the existing PCR is the reference standard, and the new method is the index test. Although PCR is the current widely accepted reference test for detecting SARS− COV-2, it does not have excellent sensitivity and specificity (Ducray et al., 2020;Ridgway et al., 2020;Zhifeng et al., 2020). Therefore, a meta-analysis based on clinical research should consider the influence of references on the analysis and interpretation of results.
At present, some systematic analyses have evaluated the diagnostic accuracy of nucleic acid detection methods in COVID-19 diagnosis, but the sensitivity observed in various studies is inconsistent. A recent metaanalysis showed that the sensitivity of RT-PCR for detection of SARS− COV-2 in nasopharyngeal swabs (NPS) and sputum samples was 73.3 % and 97.2 %, respectively (Böger et al., 2021). Mustafa et al. found that the pooled sensitivity of NAATs evaluating SARS− COV-2 was 90.4 %. Furthermore, they argue that since there is no gold standard for diagnosing coronavirus infection, any existing NAATs can be used as a reference standard (Mustafa Hellou et al., 2021). A systematic review showed that the sensitivity of RT-LAMP was 94 % in purified RNA from COVID-19 patient samples and 78 % in crude samples. And RT-LAMP was prone to false-negative results in low viral load samples (Subsoontorn et al., 2020). Although these studies reported the performance of molecular detection in SARS− COV-2 infection, few of them consider the impact of inconsistent reference standards on the results. The reference method is essential for evaluating the performance of the detection method. Comparing the new suboptimal analysis with the established but suboptimal analysis may lead to untrue conclusions about the performance of the new way (Bohn et al., 2020).
Moreover, in these meta-analyses mentioned above, few studies have explored the significance of false-negative rate (FNR) in diagnosing SARS− COV-2 infection in detail. False-positive can result in unnecessary case isolation and further testing. However, FNR may present delays with prompt isolation and treatment, leading to the spread of infection (Xiao et al., 2020). The FNR might be related to the current outbreaks growing and epidemic rebound of SARS− COV-2. Taking the epidemic situation in Shijiazhuang, China, as an example, as of January 23, 2021, three rounds of mass testing have been carried out, with a total of more than 30 million tests in 17 days. Patients with a positive result in the three tests were 354, 247, and 30, respectively. Eighty percent of them are from quarantine sites (Li, 2021). Besides the low viral load and extended incubation period, the FNR has become another important factor of concern (Kucirka et al., 2020;Walsh et al., 2020). Therefore, to put nucleic acid testing in perspective, clearly understanding the diagnostic accuracy of different detection methods is essential for diagnosis, prevention, and control of this pandemic.
This study evaluated the diagnostic accuracy of RT-PCR and RT-LAMP and the potential effects of reference methods on diagnostic accuracy. Detailed subgroup analyses, FNR, and false positive rates (FPR) were included in this meta-analysis.

Search strategy
We searched PubMed, the Cochrane Library, Web of Science, China National Knowledge Infrastructure (CNKI), and the Chinese Bio-Medical Literature Service System (Sino Med) without limits of language (the literature time limit is updated September 1; 2021). The search strategy included the following search string: "(COVID-19 OR novel coronavirus disease OR SARS− COV-2) AND (laboratory molecular test OR nucleic acid amplification tests OR RT-PCR OR RT-LAMP) AND (diagnostic)". We performed manual searches in the references lists.

Inclusion and exclusion criteria
We included clinical studies assessing the diagnostic accuracy of RT-PCR and RT-LAMP using respiratory samples. Sensitivity (Se) and specificity (Sp) were quantitatively reported. We included both casecontrol and prospective cohort studies published until September 1, 2021. Animal or in-vitro studies and case reports were excluded.
Included: (i) Diseases: COVID-19 infection or suspected cases; (ii) Type of study: Diagnostic accuracy test or comparsion between different nucleic acid assay; prospective study or retrospective study; (iii) Index test methods: RT-LAMP and RT-PCR; (iv) Reference standard: RT-PCR (currently recognized as the gold standard for detection) or the other methods (such as clinical diagnosis, sequencing, a combined result of different RT-PCR assays. Excluded: (i) Diagnostic tests for other diseases; (ii) The accuracy study of chest computed tomography (CT), serum antibody detection, and other non-nucleic acid detection methods in COVID-19 diagnosis; (iii) Case reports, in vitro cell experiments, and animal experiments; (iv) incomplete data or the inability to extract relevant data; (v) the studies that are missing in patient selection and reference method reporting were excluded.

Data collection and quality assessment
The Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) tool was used to assess the quality, the potential for bias, and the applicability of each selected study. Two review authors (DS and YB) independently screened studies and resolved disagreements with a third review author (YH). The QUADAS-2 tool evaluates four domains of bias: patient selection, index test, reference test, and flow and timing of testing. Studies considered to have a high risk of bias were not included in the meta-analysis. Two review authors (RP and YY) independently extracted study characteristics and 2 × 2 diagnostic-accuracy table data, which were checked by two authors (LM and SL).

Statistical analyses
Spearman correlation coefficient between the logarithm of sensitivity (Se) and the logarithm of (1-Sp) was calculated to detect the threshold effect. Generally, a strong positive correlation indicates the existence of a threshold effect (Leeflang, 2014). Deek's Funnel graph method was used to detect publication bias, and the result was determined by funnel plot and P value of the slope coefficient.
We performed an analysis of diagnostic trials based on the extracted data of true positive (TP), false positive (FP), false negative (FN), true negative (TN). The bivariate random-effects model of Midas command of Stata 12.0 software was used. The diagnostic performance of RT-PCR and RT-LAMP was shown in terms of summary sensitivity (Se), specificity (Sp), likelihood ratio (LR), and summary receiver operating curve (SROC), with a 95 % confidence interval (CI). The results were visually represented using forest plots.
The FNR and FPR were calculated according to the formula: We obtained summary estimates of FNR and FPR by using randomeffects meta-analysis. SROC curves were built to describe the relationship between test sensitivity and specificity. An area under the SROC close to 1 indicated an excellent diagnostic performance of the test. All analyses were performed using STATA 12.0.
This meta-analysis evaluated diagnostic test accuracy indexes, including authenticity evaluation indexes (Se, Sp, FNR, FPR) and comprehensive evaluation indexes (LR and SROC curves). Results were presented as summary values (95 % CI). The heterogeneity of the studies was established by I 2 , with I 2 > 50 % considered moderate heterogeneity and I 2 > 75 % defined as high heterogeneity. Our study conducted subgroup analyses to explore the possible sources of heterogeneity. Meanwhile, we assumed the prevalence of 10 %, 20 %, and 30 % of the population, respectively, to calculate the positive predictive value (PPV) and negative predictive value (NPV) under different prevalence rates.

Search result
A total of 2584 non-duplicated studies were obtained, and 2274 studies were excluded at the title and abstract level. Finally, 145 full-text articles were evaluated. Through further screening and evaluation of bias risk and applicability of research, 112 full texts have been excluded. Reasons for exclusion: the included study population was unqualified (P). The intervention measures were other detection methods (I). RT-PCR was a reference test, but the index test was another non-nucleic acid detection method (C). Outcome indicators are unavailable, or data cannot be extracted (O). Risk assessment shows a high risk, such as not mentioning whether or not to use the blind method. Other reasons: The number of studies updated in the later stage was insufficient for analysis, so they were excluded. When more similar files are updated later, further analysis can be carried out. Thirty-three studies were included, with 13 related studies on the diagnostic accuracy of RT-LAMP and 16 on RT-PCR (Fig. 1).

Characteristics of included studies
Characteristics of included studies are shown in detail in Table 1. Four studies assessed more than one index test to compare consistency between different tests. In order to fully present the data of these studies, we included these duplicates in our meta-analysis (Bulterys et al., 2020;    Hu et al., 2020;Smithgall et al., 2020;Zhen et al., 2020b). RT-PCR was defined as a reference method in most of the included studies. Mixed respiratory tract samples or deep sputum samples from the trachea or throat were reported in 7/33 studies. Clinical diagnosis, sequencing, and multiple RT-PCR combinations were used as the reference criteria in four studies (Bisoffi et al., 2020;Fukumoto et al., 2020;Shen et al., 2020;Zhen et al., 2020a).

Analysis of diagnostic threshold effect
Threshold effect is one of the main causes of heterogeneity in diagnostic tests. The threshold effect between studies can be determined by calculating the Spearman correlation coefficient between the Se and Sp of included studies. When Se is positively correlated with 1-Sp, it indicates a threshold effect.
The Spearman correlation coefficient of RT-LAMP is equal to -0.218 (P = 0.435). The Spearman correlation coefficient of RT-PCR is equal to -0.365 (P > 0.5). The above results indicate no threshold effect.
The positive predictive values (PPV) and negative predictive values (NPV) of RT-LAMP and RT-PCR are shown in Table 2.

Subgroup analysis
Subgroup analyses of RT-LAMP diagnostic values are shown in Fig. 5. Compared with nasopharyngeal swabs(NPS)or oropharyngeal swabs (OPS), the mixed sample of the respiratory tract showed better sensitivity (0.98) and lower FNR (0.02). Compared with a single target gene, multiple target genes has higher sensitivity (0.95) and a lower FNR (0.03). Interestingly, when the reference method was RT-PCR with multiple target genes, the sensitivity and FNR of RT-LAMP were 0.84 and 0.15, respectively; when the reference method was RT-PCR with a single target gene, the sensitivity and FNR of RT-LAMP were 0.90 and 0.11, respectively.
Subgroup analyses of RT-PCR diagnostic values are shown in Fig. 6. When the samples were NPS or OPS, the FNR of RT-PCR was 0.06, but the heterogeneity of the results was high (I 2 > 75 %). The RT-PCR with multiple target genes had higher sensitivity (0.95) and a lower FNR (0.05) compared with a single target gene. Of note, the sensitivity of RT-PCR was 0.96 (95 % CI, 0.90− 0.98) when multi-target gene RT-PCR was used as the reference method. However, when other methods (clinical diagnosis, sequencing, combination of the results of multiple detection methods) were used as the reference, the sensitivity of RT-PCR was only 0.84, and the FNR was as high as 0.15.

Discussion
A recent systematic review showed that nucleic acid testing has a high sensitivity (86 %) and specificity (96 %) in the diagnosis of acute respiratory syndrome due to coronavirus (Floriano et al., 2020). However, data on COVID-19 were not mentioned in the study of Floriano et al. In other COVID-19 related meta-analysis and systematic analyses, there is no further detailed classification of reference standards, and the heterogeneity of research results is high (Böger et al., 2021;Castro et al., 2020).
In this systematic review, the pooled sensitivity of RT-LAMP and RT-PCR was 92 % and 96 %, respecitively. The pooled specificity of RT-LAMP and RT-PCR was 99 % and 100 %, respecitively. The positive   LR and negative LR results of the two methods suggest a high diagnostic value. Moreover, the area under the SROC was above 0.9. However, the I 2 in most of these results was more than 50 %. Our study found that compared with NPS/OPS, mixed samples from multiple parts of the respiratory tract have better sensitivity and lower FNR, which is consistent with the results of other studies (Böger et al., 2021;Mustafa Hellou et al., 2021). The guideline for preventing and treating COVID-19 in China indicates that molecular diagnosis detection should use at least two gene targets to minimize the risk of false-negative . Our results also showed that RT-PCR and RT-LAMP with more than one target gene showed better diagnostic value than a single target gene. When RT-PCR was used as a reference method to evaluate the accuracy of RT-LAMP, the sensitivity of RT-LAMP with multi-gene targets was 15 % higher than that of RT-LAMP with a single gene target. Similarly, the sensitivity of RT-PCR with multiple-gene targets increased by 5%.
We can speculate that the detection method with multi-gene targets has high sensitivity and a low FNR. When RT-PCR with multi-gene targets is used as a reference method, the higher the consistency between the index test results and the reference, the higher the diagnostic value. However, from our results, the above rules are only applicable to comparing different RT-PCR methods. When other methods that are likely to correctly classify infected persons, such as clinical diagnosis and sequencing, are used as reference standards, the sensitivity of RT-PCR decreases to 87 % and FNR increases to 15 %. Therefore, RT-PCR is not a perfect gold standard. When RT-PCR (single gene target) was used as the reference method to evaluate RT-LAMP, RT-LAMP showed high sensitivity due to selecting the reference method with a low diagnostic value. Therefore, comparing RT-LAMP with established suboptimal RT-PCR may exaggerate the diagnostic value of RT-LAMP.
Due to the high heterogeneity of the above results, the conclusions need to be treated with caution. Therefore, a better reference method and research design to evaluate the diagnostic accuracy of nucleic acid detection are required. There are many influencing factors in diagnostic trials, which will result in high heterogeneity in the meta-analysis of diagnostic trials. Several factors can result in False-negative results, such as sampling, sample transportation, storage, RNA extraction, the incubation period prior to the symptom (Loeffelholz and Tang, 2020;Touma, 2020).
The advantage of this study is to analyze the diagnostic accuracy of RT-PCR and RT-LAMP more deeply based on previous evidence. Our study investigated the potential effects of different reference methods on diagnostic accuracy and performed subgroup analysis. Besides, compared to similar studies, we chose a more comprehensive evaluation index to evaluate the diagnostic accuracies of the two methods. To the best of our knowledge, this study is the first to use FNR and FPR as evaluation indexes of diagnostic accuracy-test meta-analysis. This may help to extend understanding and further explain problems in clinical diagnosis and epidemiology.
Our systematic review and meta-analysis has several limitations. Firstly, some results showed high heterogeneity. Although, the heterogeneity of analysis results can be reduced through subgroup analysis, some may still have high heterogeneity (I 2 > 50 %). This may be related to the inevitable clinical heterogeneity among studies. Secondly, we unified the reference method as RT-PCR, and stratified analysis according to different subgroups. However, different reagent manufacturers and experimental platforms will lead to inevitable heterogeneity. Besides, without a definitive reference standard, it may produce uncertain factors in the design and analysis of such consistent research (Petitti, 2001). These may explain the considerable heterogeneity of most meta-analyses on the same subject.

Conclusion
RT-PCR and RT-LAMP have high diagnostic values in the diagnosis of COVID-19. However, meta-analysis results of the sensitivity and FNR showed high heterogeneity. The accuracy of diagnostic tests is closely related to the sample type and reference criteria. RT-PCR has high sensitivity and a low FNR (pooled FNR of about 6%). The overall pooled specificity of RT-LAMP and RT-PCR was as high as 99 %. However, the sensitivity of RT-LAMP was poor, and about 12 % of true positive patients were considered negative. RT-PCR is the most commonly used reference method at present. The premise is that PCR has excellent sensitivity and specificity, but this is not the case. Therefore, choosing accurate and reliable reference methods is essential when evaluating the diagnostic accuracy of nucleic acid detection methods.

Funding statement
This work is supported by the Belt and Road Special Project of Lanzhou University (grant numbers 2018ldbrzd008).

Appendix A. Supplementary data
Supplementary material related to this article can be found, in the