{'fraction_non_alphanumeric': 0.02358755927027244, 'fraction_numerical': 0.0058667523909025155, 'mean_word_length': 5.603077739453436, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 2, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Quality assurance guidance for scoring and reporting for pathologists and laboratories undertaking clinical trial work

Abstract While pathologists have always played a pivotal role in clinical trials ensuring accurate diagnosis and staging, pathology data from prognostic and predictive tests are increasingly being used to enrol, stratify and randomise patients to experimental treatments. The use of pathological parameters as primary and secondary outcome measures, either as standalone classifiers or in combination with clinical data, is also becoming more common. Moreover, reporting of estimates of residual disease, termed ‘pathological complete response’, have been incorporated into neoadjuvant clinical trials. Pathologists have the expertise to deliver this essential information and they also understand the requirements and limitations of laboratory testing. Quality assurance of pathology‐derived data builds confidence around trial‐specific findings and is necessarily focused on the reproducibility of pathological data, including ‘estimates of uncertainty of measurement’, emphasising the importance of pathologist education, training, calibration and demonstration of satisfactory inter‐observer agreement. There are also opportunities to validate objective image analysis tools alongside conventional histological assessments. The ever‐expanding portfolio of clinical trials will demand more pathologist engagement to deliver the reliable evidence‐base required for new treatments. We provide guidance for quality assurance of pathology scoring and reporting in clinical trials.


Introduction
A Quality Assurance in Clinical Trials Workshop was convened by the UK National Cancer Research (MHRA), industry and pathology. Four subgroups were formed to tackle issues around regulation: training: trial oversight: and scoring and reporting. The subject areas were researched by the subgroup members prior to the workshop and presented by the subgroup leads for general discussion by the workshop participants. This article provides an overview of the discussions around scoring and reporting of pathology parameters in clinical trials. Discussions around the benefits and challenges in using digital pathology and image analysis were wide-ranging and will therefore be presented in a separate article. The workshop formulated 'practice points' to help pathologists navigate the development, set up and delivery of clinical trials and these are placed in italics at the end of each section and compiled in Table 1.
Scoring and reporting of pathology parameters is an interpretive skill; consequently there is an acknowledged subjective variation in clinical practice from pathologist to pathologist, even within the same centre. For clinical trials, the challenge is to control for variations in individual pathologist practice to limit the introduction of bias and errors. Common pathology parameters used in clinical trials include: diagnosis; grading; staging; biomarker scoring; pathological complete response (pCR); resection margins and recurrence.
Diagnosis, grading and staging are routinely carried out in clinical practice following well-established guidelines (e.g. Royal College of Pathologists, College of American Pathologists, Union for International Cancer Control, American Joint Committee on Cancer). The information can be collected from participating sites using a carefully designed 'pathology case report form' (Pathology CRF). Central review of at least a proportion of specimens is desirable for quality assurance purposes, but the extent of review will depend on the aims and objectives of the trial, diagnostic confidence, reproducibility of grading schemes and the ease of application of staging systems.
Immunohistochemistry, in situ hybridisation or molecular profiling of tumours may be used for trial entry, patient stratification and randomisation into treatment arms. If a trial is multicentre, it must be considered whether testing will be carried out at the participating sites or co-ordinated by a central laboratory. Generally, it is preferable that biomarker testing is carried out at the participating sites as they benefit from proximity to patient care and this mitigates against delays in the patient pathway. In the case where laboratory tests are not generally available or are specialised tests developed for the trial, then a central laboratory with the requisite expertise and appropriate level of accreditation is necessary to run the trial effectively. It is important to consider realistic turnaround times when considering central laboratory testing and to manage the expectations of the participating sites and trial co-ordinators. Trial sample logistics, laboratory capacity and likelihood of repeat tests need to be considered. Whether tests are delivered at participating sites or a central laboratory, quality assurance measures need to be considered and implemented.
It is well known that the results of laboratory tests are influenced by a myriad of variables (Table 2) [1]. Pre-analytical variables are those associated with preservation and preparation of tissue prior to testing. Analytical variables are specific to the test carried out. Post-analytical variables, particularly associated with

92
M Robinson et al immunohistochemistry, are a consequence of subjective assessment by the pathologist, which is at best semi-quantitative, but variability can be estimated by measuring the degree of inter-observer and intraobserver agreement. Molecular testing of tissue homogenates using 'grind and bind' techniques benefit from quantitative results, but each specific assay has characteristic technical challenges and they lack the morpho-molecular features of tests carried out on intact tissue sections [2].

Pathology reporting in clinical trials
Ideally, assessment of pathology-based clinical trial endpoints should be masked or blinded to randomisation and clinical outcomes. For exploratory outputs, a staged approach may be more appropriate, such as that proposed by the Society for Toxicological Pathology [3]. The first stage is an un-blinded comparison of treated and control specimens to identify consistent changes and to develop scoring criteria. This strategy facilitates identification of subtle, treatment related findings that can be consistently differentiated from those that occur in controls [4]. An independent pathology peer review with targeted blinding may also help minimise bias [5].
Scoring and reporting should be carried out blinded to treatment allocation and clinical outcomes.

Diagnostic drift and chronological bias
Diagnostic drift and chronological bias should be considered when setting up a clinical trial. Diagnostic drift is a gradual change in nomenclature, grading of lesions, or scoring of a biomarker within a single study over time. It is a source of inconsistency that can negatively affect detection of treatment-related changes or the determination of 'no-effect' levels [3]. Chronological bias is defined as the evolutionary process of a grading system, whereby more sensitive and specific criteria for grade assignment are clarified, learned and disseminated. As pathologists gain experience, subtleties of the evolving system are applied to interpretation of tissue sections [6]. This is also a source of inconsistency that can negatively affect detection of treatmentrelated changes or the determination of the 'no-effect' levels [4]. Plans for monitoring diagnostic drift and chronological bias should be considered for trials projected to recruit over several years. Comparison of pathology data at planned intervals throughout the life of the study, for example at the beginning, middle and end of the study, is recommended.
The effect of diagnostic drift and chronological bias should be considered throughout the trial, in particular if scoring and reporting is carried out over several years and/or if new reporting guidelines are published.

Double reporting and central review
Double reporting to verify pathological parameters such as diagnosis, grade, stage and biomarker scores is recommended, but may not be feasible in real time. Pathology review of cases should be considered as part of the trial design. The extent of the review process will depend on the number of cases, the complexity of the scoring, and the contribution of pathology parameters to the trial endpoints. Statistical advice should be sought when designing the review strategy. Central review by a lead pathologist or a group of pathologists should be completed before final data analysis and trial publication. For example, Speight et al (2015) described a system for review of cases from patients with oral epithelial dysplasia that involved four pathologists with adjudication of disagreements towards a consensus diagnosis as the gold standard for clinical  [7]. Central pathology review may be 'blinded', meaning independent assessment without knowledge of the participating site data or 'un-blinded' where the original pathology report or pathology CRF is available and data are simply checked for any disagreements.
Double reporting and/or central review should be considered to increase confidence in the pathology data.

Pathology working groups
Pathology working groups have been used in some trials to good effect. For example, ProtecT (Prostate Testing for Cancer and Treatment), a randomised controlled trial comparing active monitoring, radiotherapy and radical prostatectomy in patients with localised prostate cancer, employed a Pathology Working Group that oversaw the pathology aspects of the trial and published impactful pathology-specific papers from the trial [8,9]. Such groups enhance the quality of a study by actively engaging pathologists in the development of trial protocols, providing training, formulating pathology standard operating procedures (SOPs), conducting central review, resolving discrepancies and disseminating the trial findings to the pathology community.
The formation of a 'Pathology Working Group' should be considered to oversee the pathology aspects of a study.

Providing quality assured pathology tests
To ensure that pathology laboratory tests are scored and reported in a consistent manner it is recommended that a systematic process is established comprising, in sequence, education, training and calibration.

Education and training
Pathologists are most likely to be asked to provide immunohistochemistry-based tests for clinical trials. Such tests may form part of the recommended datasets for the histopathological reporting of cancer, in which case the expertise is likely to be already established, for example the scoring of oestrogen and HER2 receptors in breast cancer [10,11]. Furthermore, they are likely to be supported by external quality assurance programmes (Table 3) [12]. For novel biomarkers, where expertise is not established, it is essential that participating pathologists understand the clinical context of the test and the role of the test in the trial; screening, recruitment, stratification or randomisation.
Knowledge of the performance characteristics of the test is essential in order to recognise sub-optimal tests. For example, assessment of external analyte controls (same-slide based proprietary cell line controls or characterised tissue controls) and review of expected staining of internal controls are required before scoring the tumour. Trial-specific training may be delivered by face-to-face site meetings, establishing trial-specific pathology working groups or using on-line training modules [13]. Such meetings can also be used to collect feedback from the participating pathologists on the SOPs associated with tissue collection, processing, staining and scoring. In some circumstances, particularly with more complex SOPs, it is best practise to run trial-specific protocols, prior to the trial opening, to identify any operational problems.
Training requirements for pathologists participating in clinical trials need to be addressed and will depend on whether the parameter is an established clinical test or a novel biomarker.

Immunohistochemistry
Guidance on validation of immunohistochemistry assays in a clinical service is already available [14]. This guidance can be followed when incorporating immunohistochemistry into a clinical trial. The assay must be assessed for accuracy and analytical sensitivity/specificity. Once validation is completed, the assay should be regularly monitored and performance against known external positive controls assessed [14].
In the UK, in the setting of a clinical trial, if immunohistochemistry forms part of the primary or secondary outcomes, then this must be performed to a standard to satisfy the MHRA, who inspect laboratories Auto-stainers should be used for immunohistochemistry in preference to manual staining.
Staining should be carried out in an ISO15189: 2012-accredited laboratory or laboratory working to GCP standards and is mandatory for primary or secondary endpoints of CTIMP clinical trials.

Scoring of immunohistochemistry
Numerous scoring methods have been devised for the assessment of immunohistochemical staining. The majority are based on the assessment of two parameters: intensity of the staining and proportion of the tumour that is stained (Table 4) [15]. Occasionally the scoring system includes an assessment of the tumour and the immune cells; an example is PD-L1 testing in non-small cell lung cancer [16]. Scoring systems associated with recognised predictive and prognostic markers should be scored according to guidelines [10,11,17]. For exploratory tests, researchers should investigate the literature and implement validated scoring systems or devise a scoring system to adequately capture the data and to facilitate the development of clinically relevant 'cut-offs'. Tumour heterogeneity is recognised as an important variable and needs to be considered when devising scoring systems. Whilst it is recognised that pathologist scoring is semi-quantitative and subjective, measurement of inter-observer and intra-observer agreement provides evidence of the reproducibility of the scoring system and increases the degree of confidence in the data [18]. For example, a study that examined Ki-67 (MIB-1) scoring across eight laboratories and a central reference laboratory demonstrated high intra-laboratory reproducibility (intraclass correlation = 0.94; 95% CI 0.93-0.97), but only moderate inter-laboratory reproducibility (intraclass correlation = 0.71, 95% CI = 0.47-0.78). Factors contributing to inter-laboratory discordance were tumour region selection, counting method and subjective assessment of staining positivity. Formal counting methods gave more consistent results than visual estimation [19]. There is considerable interest and investment in developing quantitative image analysis for scoring immunohistochemistry [1].
Measures to assure confidence in the reproducibility of pathologist scoring should be considered, for example by reporting levels of inter and intra-observer agreement.

Filtering
Individual pathologists have different thresholds for observing and recording incidental morphological changes, background staining or unexpected staining patterns, called filtering. Recording such data, particularly for novel biomarkers, may be important as the information contributes to an understanding of the characteristics of a particular test and informs biomarker development and refinement.
The pathologists in the trial should agree what is considered background staining, and thus filtered out of pathologist scoring, and this should be kept under review during the trial.

Companion and complementary diagnostic tests
Increasingly, the development of new drugs requires specific diagnostic tests to guide treatment, called companion and complementary diagnostic tests. Companion tests are mandatory prior to administering a targeted drug, whereas complementary tests are used to guide patient selection for treatment, but are not used as 'gate-keepers' for access to the drug. The rationale Table 4. Scoring systems for immunohistochemistry [15] Scoring method for recommending such tests is based on the principle of precision medicine: delivering the right drug to the right patient at the right time, but is also driven by the requirements of regulatory bodies such as the US FDA and the UK MHRA. The paradigm for companion testing is breast cancer where HER2 receptor testing (e.g. HercepTest, Agilent Dako, Glostrup, Denmark) is required prior to treatment with Trastuzumab. Such diagnostic tests are manufactured to high standards and satisfy the requirements for an in vitro diagnostic (IVD) medical device, accredited by regulatory bodies. Furthermore, UK National External Quality Assurance Scheme (NEQAS) immunocytochemistry (ICC) and in situ hybridisation (ISH) have shown that laboratories using IVD tests outperform those using laboratory developed assays [20]. One of the issues around the use of IVDs is that the tests tend to be expensive and the manufacturers' instructions need to be followed precisely to produce a valid test result. Typically, the instructions specify the use of a specific proprietary auto-stainer, which may not be available in participating laboratories. The problem can be resolved by setting up central laboratory testing at a site with the designated staining platform that can deliver quality assured tests.
Companion and complementary diagnostic tests should be used in preference to laboratory-devised assays. Manufacturers' instructions must be followed to produce a reliable result.

Estimates of uncertainty
Scoring immunohistochemistry may be subject to 'estimates of uncertainty of measurement'. ISO 15189:2012 states that 'the laboratory shall at least attempt to identify all the components of uncertainty and make a reasonable estimation, and shall ensure that the form of reporting of the result does not give the wrong impression of uncertainty'. This applies to numerical counts of immunohistochemically positive cells and working examples are provided in guidelines provided by the UK Royal College of Pathologists [21]. The recommendations include consideration of the best methods to achieve clinically reliable measurements, ensuring that these are defined in SOPs and working to ensure that the measurement procedures are consistent between pathologists [21].
Estimates of uncertainty, defined in ISO 15189:2012, should be made for clinical trial tests where possible.

Calibration
It may be necessary to calibrate the laboratories and pathologists prior to opening a trial. This can be achieved by providing a 'test set' of cases and compiling the results across the centres and providing feedback. This can be an iterative process aimed at achieving a pre-specified level of performance or more formal accreditation of the laboratory for the trial. Typically, accredited laboratories would have high levels of agreement with a reference laboratory or demonstrate high levels of agreement with other laboratories taking part in the trial. Statistical tests, such as Cohen's Kappa coefficient (ĸ) [22] or intraclass correlation coefficient, can be used to measure interlaboratory or inter-observer agreement. In the case of Cohen's Kappa coefficient to assess categorical variables, the acceptable level of agreement is controversial. Some researchers consider a ĸ of >0.6 (moderate agreement) acceptable, whereas others stipulate a ĸ of >0.8 (strong agreement) [22]. A statistician should be consulted for advice on the most appropriate statistical tests to use. Ideally, the statistician should analyse the data independently of the pathologist(s) to ensure objective assessment.
The requirement for pathologist calibration should be considered prior to the trial opening.

External quality assurance
For established biomarkers, where external quality assurance programmes already exist (e.g. UK NEQAS for ICC and ISH), trial laboratories should participate in such schemes and demonstrate satisfactory performance (Table 3) [12].
If an external quality assurance scheme (e.g. UK NEQAS ICC & ISH) is available for the trial test then trial laboratories should participate.

Genomic testing
Factors such as acceptable testing platforms, nucleic acid extraction kits, modality of testing and the thresholds for positive results need to be set out in SOPs. Fresh-frozen tissue is the gold standard for techniques where nucleic acid quality requirements are high, for example whole genome sequencing. Formalin-fixed paraffin-embedded samples may be acceptable but are subject to the numerous pre-analytical variables listed in Table 2 [1,23]. If an external quality assurance scheme for example, the UK NEQAS Molecular Genetics programme exists for a particular test, then laboratories should participate [24]. Pathologists should review trial-specific patient information sheets and consent forms to ensure that laboratory tests

Pathological complete response
Pathological estimates of residual disease following trial-specific interventions, termed 'pathological complete response', are being utilised to expedite the reporting of clinical trials, where previously only extended clinical follow up has been used [25][26][27][28]. It is important that the participating pathologists have an understanding of the definition of pCR; for example in breast cancer trials, variable definitions have been used allowing or excluding ductal carcinoma in situ from the definition, which influences trial outcomes [29]. Nevertheless, central pathology review of a phase 3 neoadjuvant breast cancer trial (ARTemis) has shown good concordance of pCR with participating centre assessment, even in the absence of guidelines and a trial-specific reporting proforma [30]. For colorectal cancer, the UK Royal College of Pathologists reporting guidelines specify that the entire scar should be embedded and three deeper levels cut on each block prior to calling a pCR. These principles should be followed in trials and it is recommended that all pCR cases are centrally reviewed. It is also important to specify whether residual nodal disease should contribute to pCR, as assessment of mesorectal lymph nodes has been shown to reduce the rate of pCR [31]. Detailed SOPs are required to standardise block selection and sectioning protocols to ensure consistent assessment of patient specimens across the trial sites [26]. Similar principles can be applied to other morphological parameters such as defining and measuring resection margins in surgical trials and processing sentinel lymph node specimens.
Clear definitions of reporting of pathological complete response and SOPs for standardised assessment need to be agreed and established prior to the trial opening.

Bio-resource for translational research
Tissue collected and curated as part of a clinical trial (frozen tissue and/or formalin-fixed paraffin-embedded tissue) is associated with high quality clinical data and pathological classification. Consequently, it is an important legacy that can be used for translational research; for example, the tissue can be used to develop and refine predictive tests. At trial inception, pathologists should actively promote tissue collection and incorporate the costs into the trial grant application or formulate applications for a companion study. The trial documentation, specifically the consent form and patient information sheet, must be worded appropriately. If samples are to be kept beyond the end of the trial, then consent needs to be enduring and ideally generic. Samples may be moved into a biobank with the correct consent provisions, ensuring that the relevant regulations are followed; for example, in England, tissue must be moved under an Human Tissue Authority (HTA) licence and meet HTA standards when the research ethics approval for the study expires.
Pathologists should be actively involved in developing trial specific bio-resources.

Conclusion
Our Quality Assurance in Clinical Trials Workshop highlighted the importance of ensuring reproducible scoring and reporting of pathology parameters in clinical trials. Key considerations include pathologist training and calibration, and measurement of inter-observer variation. There are opportunities to evaluate image analysis systems in clinical trials, alongside conventional histological assessment, to moderate 'uncertainty of measurement'. In any case, pathologists' engagement in clinical trials is essential to help deliver the reliable evidence-base required for new treatments and robust pathology processes are vital to the credibility of studies.
Pathologists' contributions to clinical trials are wide ranging and should be recognised and appropriately funded.
Early pathologist engagement and input into trial design is essential.