{'fraction_non_alphanumeric': 0.03229855242239143, 'fraction_numerical': 0.018617688896326666, 'mean_word_length': 5.052924791086351, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 6, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Sensitivity and Specificity of Rapid Diagnostic Tests for Hepatitis C Virus With or Without HIV Coinfection: A Multicentre Laboratory Evaluation Study

Abstract Background Hepatitis C virus (HCV) screening is critical to HCV elimination efforts. Simplified diagnostics are required for low-resource settings and difficult-to-reach populations. This retrospective study assessed performance of rapid diagnostic tests (RDTs) for detection of HCV antibodies. Methods Two lots of 13 RDTs were evaluated at 3 laboratories using archived plasma samples from 4 countries (Nigeria, Georgia, Cambodia, and Belgium). HCV status was determined using 3 reference tests according to a composite algorithm. Sensitivity and specificity were evaluated in HIV-infected and HIV-uninfected populations. Operational characteristics were also assessed. Results In total, 1710 samples met inclusion criteria. In HIV-uninfected samples (n = 384), the majority of RDTs had sensitivity ≥98% in 1 or both lots and most RDTs had specificity ≥99%. In HIV-infected samples (n = 264), specificity remained high but sensitivity was markedly lower than in HIV-uninfected samples; only 1 RDT reached >95%. The majority of HIV-infected samples for which sensitivity was low did not have detectable HCV viral load/core antigen. Interreader variability, lot-to-lot variability, and rate of invalid runs were low for all RDTs (<2%). Conclusions HCV RDTs should be evaluated in the intended target population, as sensitivity can be impacted by population factors such as HIV status. Clinical Trials Registration NCT04033887

In 2015, the number of people with chronic hepatitis C (HCV) infection worldwide was estimated at 71 million [1]. However, only around 20% of people with HCV are aware of their HCV status [1]. HCV screening is critical to the success of HCV elimination targets, but in low-and middle-income countries (LMICs), where standardized laboratory tests are expensive and often not covered by public health systems, screening of at-risk populations for HCV infection remains very limited [2]. The burden of HCV in LMICs is particularly high, representing over 70% of the global total [3]. As such, the World Health Organization (WHO) strategy to eliminate HCV has highlighted an urgent need for simplified diagnostic tests for use in low-resource settings, as well as for difficult-to-reach populations in high-income countries, such as people who inject drugs [4].
Screening for HCV is performed through the detection of HCV-specific antibodies. WHO guidelines recommend the use of a single quality-assured serological in vitro diagnostic test, either a laboratory-based immunoassay or a rapid diagnostic test (RDT) [5]. For many LMICs, where equipped laboratories and trained staff are limited, RDTs may be most appropriate, as they are quick and easy to perform without the need for laboratory equipment. RDTs have proved effective in other disease areas; for example, the wide availability of low-cost RDTs for the diagnosis of human immunodeficiency virus (HIV) has substantially increased access to testing, resulting in more than 600 million people being tested for HIV in LMICs from 2010 to 2014 [6].
The lack of quality-assured RDTs for HCV serology testing has been identified as an important barrier to large-scale access to HCV diagnosis [2]. While a number of HCV RDTs are commercially available, many do not have quality assurance status (eg, stringent regulatory authority approval or WHO prequalification [7]). Additionally, data on the quality and performance of many tests are limited, especially in LMICs. WHO recommendations on performance criteria for procurement of in vitro diagnostics for HCV, which also serve as guidance for WHO prequalification, recommend a sensitivity of ≥98% and a specificity of ≥97% for HCV serology RDTs in plasma or serum specimens [8]. Data on sensitivity and specificity of existing RDTs and RDTs in development can help to determine whether additional tests may be suitable for WHO prequalification, and results of independent performance evaluations can support countries in their choice to procure tests that meet international performance criteria.
Furthermore, some studies have noted a potential negative impact of HIV coinfection on the sensitivity of some HCV RDTs [9][10][11]. This may be due to the compromised immune system of people living with HIV limiting the production of anti-HCV antibodies; data on RDT performance by CD4 count (an indicator of immune status in HIV-positive people) have been identified as a research gap for HCV serology testing [5]. Given the high burden of HIV in LMICs, and the substantial proportion of people with HCV and HIV coinfection worldwide (approximately 2.3 million) [1], understanding the effect of HIV status on HCV RDT performance will be crucial to HCV elimination efforts.
The objective of this study was to evaluate the performance of a range of HCV RDTs using clinical samples collected from different geographic regions, as well as from HIV-infected individuals, in order to identify tests that could be used for HCV screening in LMICs or difficult-to-reach populations.

Study Design
This was an observational, retrospective, multicenter laboratory evaluation of 13 HCV RDTs (NCT04033887; Table 1). Nine RDTs were on-market products, 1 RDT (HCV-only Ab Test; Biosynex SA) had its configuration adapted to only evaluate the HCV line (the on-market product configuration is a triplex test with additional lines for the detection of HIV antibodies and hepatitis B virus surface antigen; the evaluated version lacked the test lines for HIV and hepatitis B), and 3 RDTs were still in late-stage development at the time of the study (defined here as prototype).
Tests were evaluated at 3 laboratories: the Nigerian Institute of Medical Research (Lagos, Nigeria), the Lugar Center at the National Center for Disease Control and Public Health (Tbilisi, Georgia), and the Institute of Tropical Medicine HIV/Sexually Transmitted Diseases (STD) Reference Laboratory (Antwerp, Belgium). Testing was performed on randomly selected locally archived frozen plasma samples from these 3 laboratories. Additionally, samples obtained from the Sihanouk Hospital Center of Hope (Phnom Penh, Cambodia) were tested at the Institute of Tropical Medicine HIV/STD Reference Laboratory; samples were frozen in Cambodia and remained frozen throughout transportation to Belgium. All sites received approval for the study from the respective institutional review boards. Testing was performed between September 2018 and March 2019.
All samples were ethylenediaminetetraacetic acid (EDTA)treated plasma samples taken from people aged ≥18 years, with a minimum volume of 1.5 mL and known HIV status. Information on HCV and HIV treatment status of the sample donors was available. No further information on the characteristics of the sample donors were collected as part of this study. Samples were nonhemolytic, had <3 freeze-thaw cycles, and had been stored at or below −70°C. Samples were collected between 2008 and 2018 (92% collected between 2014 and 2018). Samples were excluded if generic consent for further use was missing. Prior to commencement of testing, each site prepared small aliquots from the master samples to eliminate the need for multiple freeze-thaw cycles. HCV antibody status was determined using 3 reference tests, of which 2 were WHO prequalification approved enzyme immunoassays (EIA; Murex Anti-HCV version 4.0, DiaSorin S.A., and INNOTEST HCV Ab IV, Fujirebio Europe) and 1 was a line immunoassay (LIA; MP Diagnostics HCV blot 3.0, MP Biomedicals). A signal to cutoff ratio of ≥1 (based on the measured optical density) was used for the EIAs; interpretation of LIA results was performed according to manufacturer instructions. HCV antibody status of each sample was determined according to a composite algorithm incorporating the results of all 3 reference tests (Supplementary Table 1). A similar algorithm has previously been used in WHO prequalification evaluation protocols, although the WHO algorithm does not require LIA confirmation for samples testing negative on both EIAs [12].

Outcomes
The primary outcomes were point estimates of sensitivity and specificity with 95% confidence intervals of the 13 HCV RDTs in HIV-infected and -uninfected samples. Secondary outcomes included sensitivity and specificity of the 13 RDTs in the overall population (regardless of HIV status), interreader variability, lot-to-lot variability, and the rate of invalid runs. Exploratory outcomes included point estimates of sensitivity and specificity with 95% confidence intervals of the 13 RDTs in HIV-infected and -uninfected samples with active HCV infection measured by the presence of detectable HCV viral load (VL) or core antigen (cAg) in the sample. Analysis of test performance by CD4 count range (<200 cells/mm 3 [severely immunocompromised], 200-500 cells/mm 3 [immunocompromised], or >500 cells/mm 3 [not immunocompromised]) in HIV-infected samples and by HCV genotype in HIV-uninfected and -infected samples was also performed.

RDT Performance Assessments
Each sample was tested on 2 independently produced lots of each RDT and each result was read and recorded by 3 independent readers. RDT results were interpreted according to the manufacturer's instructions. Samples were scored as either positive (reactive), negative (nonreactive), or invalid on each RDT based on the concordance of at least 2 out of 3 reader results. For all samples that were scored invalid, a repeat test was performed once on the same lot.
Operators/readers of the RDTs were blinded to the results of the reference standard tests. The sequence in which samples were tested was varied for each RDT to avoid bias related to recognition patterns. Operators and reader sequences were also varied.

Statistical Analyses
For an average sensitivity of 85% and specificity of 80%, a minimum sample size of 400 for sensitivity analyses and 502 for specificity analyses was required to obtain point estimates with a precision of ± 5% and power of 80% to obtain a confidence interval with total width of 10% or less [13].
Point estimates were obtained, with 95% confidence intervals based on Wilson score method, for sensitivity and specificity. Interreader variability was assessed by Fleiss kappa coefficient (κ) (agreement was defined as concordance between 2/3 or 3/3 results for each RDT). Lot-to-lot variability was evaluated by assessing performance in each lot using final valid RDT outcomes (excluding repeatedly invalid results), and the rate of invalid runs was calculated as the ratio between runs marked as invalid and the total number.

Sample Characteristics
Of 1864 samples selected, 1710 met inclusion criteria. In total, 648 samples were HCV antibody positive, of which 264 were also HIV positive. Of the 852 HCV antibody-negative samples, 626 were HIV positive and 226 were HIV negative. Two hundred and ten samples had indeterminate HCV status due to discrepancies between EIA and LIA results or indeterminate LIA results and were excluded from further analyses as per the composite reference standard algorithm ( Figure 1). Although the sample size was not as large as was estimated to be required based on the previously stated test performance assumptions, based on the average test performance observed in this study, the sample size allowed for ≥ 80% power with a precision of ± 5 in all subgroups (Supplementary Table 2).
The numbers of samples with genotype, CD4, count and HCV VL/cAg availability, and the country of sample origin for each sample type, are shown in Table 2. The majority of genotyped samples were of HCV genotype 1, 1a, or 1b (63.2% of HIV-uninfected and 54.2% of HIV-infected samples), followed by genotype 3 in HIV-uninfected samples (31.6%) and genotype 6 in HIV-infected samples (22.9%). The majority of HIVpositive samples had CD4 counts greater than 200 cells/mm 3 (>93%). The majority (89%) of HCV-infected and HIV-infected samples were from patients receiving treatment for HIV at the time of sample collection (HIV treatment status was known for 256 of 264 HCV-positive and HIV-positive samples). None of the samples from Nigeria, Cambodia, or Georgia were from people receiving treatment for HCV; of the Belgian samples, 107 were from people who had never received treatment for HCV, 10 were from people who were on active interferon treatment, 7 were from people who had previously received interferon treatment, and 2 had no treatment information available.

Sensitivity and Specificity
In the samples from HIV-uninfected patients, most RDTs showed high sensitivity, with the majority reaching ≥98% in 1 or both lots (Figure 2A and Table 3). The large majority of tests showed a specificity of ≥99% and several reached 100% ( Figure 2B and Table 3). In HIV-infected samples, sensitivity was markedly lower than in HIV-uninfected samples, with only 1 RDT reaching >95% (Prototype DPP HCV; Chembio Diagnostic Systems) ( Figure 2C and Table 3). For the large majority of RDTs, confidence intervals between HIV-uninfected and -infected samples did not overlap. Specificity was comparatively high in HIV-infected samples, with only 4 RDTs showing a specificity of <97% in at least 1 lot ( Figure 2D and Table 3). In the combined sample set of HIV-uninfected and HIV-infected samples, results reflected the lower sensitivity for most RDTs, and lower specificity for some RDTs, observed in the HIVinfected samples (Table 3).
False negatives were distributed across 86 different samples. Of these, only 26 (30.2%) had genotype information available, with the most common genotype being genotype 6 (n = 9, 34.6%). The distribution of false negatives per CD4 count range (<200, 200-500, and >500 cells/mm 3 ) in HIVinfected samples showed that false negatives occurred at a similar frequency in all CD4 count ranges (Supplementary Table 3).
To evaluate whether RDT sensitivity was associated with detectable HCV VL/cAg, point estimates of sensitivity were calculated for HIV-uninfected and HIV-infected samples only for samples with detectable HCV VL/cAg. In HIV-uninfected samples with detectable HCV VL/cAg, sensitivity increased moderately compared with the overall sample set, while in HIV-infected samples with detectable HCV VL/cAg, sensitivity increased markedly compared with the overall sample set ( Figure 3 and Supplementary Table 4). The majority of confidence intervals did not overlap between HIV-infected samples with detectable HCV VL/cAg and all HIV-infected samples (Figure 2A and 2C and Figure 3A and 3B).

Operational Characteristics
There was a very high concordance among readers in terms of interreader variability, with a coefficient of agreement ≥95% for all RDTs and lots. Furthermore, there was a high percentage of agreement between lots per RDT, with 10/13 RDTs achieving an agreement of >98% between both tested lots. Invalid runs were uncommon; 11/13 RDTs generated no or only very few invalid results during the first run and none during the repeat (Table 4).

DISCUSSION
To our knowledge, this is the first study to evaluate the performance of HCV RDTs using a large number of samples representing different geographical regions and with a substantial proportion of HIV coinfected samples. As such, our findings provide valuable insights into HCV RDT performance on archived plasma samples and highlight a number of areas for future study.
WHO guidance on performance criteria for in vitro diagnostics for HCV recommends a sensitivity of ≥98% and a specificity of ≥97% for HCV serology RDTs [8]. In HIV-uninfected plasma samples in this study, the performance of the 13 RDTs was high; all tests met the WHO specificity criteria and 11 of 13 met the sensitivity criteria for 1 or both lots. This is consistent with previous studies demonstrating high sensitivity and specificity of the SD Bioline [10], First Response HCV Card Test [14], and OraQuick HCV [9,10,15] in plasma samples, and high performance of a number of the RDTs in other sample types including serum and oral fluid [9,10,[16][17][18][19]. In 2 systematic reviews of HCV RDTs that included studies with varying designs, references and sample types, overall pooled sensitivity was 98%-99% [17,19]. These findings suggest that a number of the RDTs tested may be suitable for in-country use in the HIV-uninfected population.
In HIV-infected samples, however, while specificity remained high (12 of 13 RDTs met the WHO specificity criteria for 1 or both lots), none of the tests evaluated met the WHO sensitivity criteria. The fact that sensitivity improved in the subset of HIVinfected samples with detectable HCV VL/cAg suggests that the reduced sensitivity in HIV-infected samples overall may have been due to low HCV antibody titers. However, the reasons for low HCV antibody titers in HIV-infected samples are unclear, as CD4 counts were generally high, suggesting that the sample donors were not severely immunosuppressed. Other studies have noted declines in HCV antibody levels following treatment-induced or spontaneous HCV clearance in HIVinfected men [20,21]. In general, our observation of lower HCV RDT performance in HIV-infected individuals is consistent with observations made in other studies, in which 1 or more of the evaluated RDTs showed poor sensitivity in samples from HIV-positive individuals [9][10][11]. The reasons for lower sensitivity in HIV-positive samples in these studies also remains unclear. More detailed information on HCV VL/cAg, time of coinfection, and initiation of and adherence to HIV treatment should be collected in future studies, in order to further assess the impact of HIV status on RDT performance.
Of the 71 million people worldwide with chronic HCV infection, 2.3 million are also infected with HIV [1]. As such, good RDT performance in HCV and HIV coinfected people is essential, particularly in LMICs where the burden of both diseases is high [3,22]. However, while the false negatives observed in HIV samples in this study are technically concerning, from a clinical perspective, it is reassuring that the diagnostic performance of the evaluated RDTs improved in HIV-infected samples with detectable HCV VL/cAg. HCV VL or cAg testing is used to confirm viremic infection in people who test positive for HCV antibodies [5], thus these samples represent patients who had active HCV infection and are ultimately in need of treatment. As RDT performance was high regardless of HIV status in these samples, the impact of HIV infection on test performance may not be that dramatic.
The majority (69.7%) of samples that were false negative in at least 1 lot of any RDT in this study did not have genotype information available, making it difficult to associate the occurrence of false negatives with any particular HCV genotype. Notably, 34.6% of all samples giving at least 1 false negative were of HCV genotype 6. Given the relatively low total number of genotype 6 samples (41 out of a total of 293 samples with genotype information available), this could potentially have been a contributing factor to the high number of false-negative samples. However, verification of this by statistical analysis was not possible due to the aforementioned low number of samples with genotype information available.
The WHO guidance on performance criteria for HCV serology RDTs recommends an interreader variability and device failure rate of ≤5% [8]. All of the RDTs evaluated in this   providing that they are first evaluated in the intended target population to determine whether sensitivity is impacted by population factors. A number of populations are commonly targeted for HCV screening, including sex workers, men who have sex with men, people who inject drugs, and people living with HIV [1,23]. Procurement of high-performance RDTs will be key to the improvement of HCV testing services for these key populations. Although we did not collect data on sample donor characteristics, it is likely that samples from these target groups were tested in our study, given the countries included. For example, Georgia has one of the highest prevalences of injection drug use globally, with up to 40% of HCV infections attributable to injection drug use [24]. Additionally, in some countries HCV screening is indicated for the general population, as a result of historical unsafe medical practices [25], as is the case in Nigeria [26] and Cambodia [27].
Limitations of this study include the uneven geographical distribution of sample types. Sensitivity in the HIV-uninfected population was primarily assessed in samples originating from Georgia and Nigeria, while sensitivity in the HIV-infected population was assessed almost exclusively in samples from Cambodia and Belgium. This makes comparisons between sensitivity in the HIV-uninfected and -infected populations challenging. We cannot exclude the possibility that differences in population characteristics, such as different types of HIV/HCV risk groups, impacted the results. Notably, Belgium (from which 120 [14.7%] samples were obtained) is a high-income country, thus population characteristics such as HIV prevalence or HCV cohort may not be comparable to those of LMICs.
While we cannot exclude the possibility that differences in storage conditions between countries had an effect on sample quality, evidence suggests that antibodies remain stable in frozen samples for several years and after multiple freeze-thaw cycles [28][29][30][31]; furthermore, we minimized any potential impact by only including samples that appeared nonhemolytic upon visual inspection. A further limitation is the low number of HCV-negative and HIV-negative samples compared with HCV-negative HIVpositive samples, which may have influenced specificity in the overall population. The impact of this was likely minor, however, as most of the RDTs performed well in both study populations.
The design of the composite reference standard led to 210 samples being excluded from the study. It is possible that inclusion of these samples would have affected the sensitivity and specificity estimates. This study did not take into account the impact of treatment for HCV, although only a small number of samples (n = 17) were from people who were receiving or who had previously received interferon treatment. Additionally, the HCV-negative samples used may not have precisely represented the target populations for HCV serology testing, leading to patient bias. Finally, tests were performed by well-trained laboratory personnel using archived samples, thus this study does not represent a real-world setting. The performance of the RDTs in primary or community care settings using prospectively collected fresh samples is yet to be established.  In conclusion, the findings from this study show that a number of available HCV RDTs may be suitable for WHO prequalification and use in HCV screening programs in LMICs. However, HCV RDTs should always be evaluated in the intended target population, as sensitivity can be impacted by population factors such as HIV status. Any evaluation panels used for assessment of HCV RDTs should contain HIV-positive samples. These findings serve as a valuable baseline to investigate RDT performance in prospectively collected whole blood samples in the intended use settings. This will yield further insights into the robustness of the RDTs when used in primary health care settings by local health workers and tested on the most common sample type used for RDTs.

Supplementary Data
Supplementary materials are available at The Journal of Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.