{'fraction_non_alphanumeric': 0.030359257797078563, 'fraction_numerical': 0.02554283458349783, 'mean_word_length': 5.1423375363724535, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A refined cell-of-origin classifier with targeted NGS and artificial intelligence shows robust predictive value in DLBCL

in a large cohort of 418 DLBCLs. Genetic and transcriptional data obtained by RNA-Seq in a single run were explored by state-of-the-art arti ﬁ cial intelligence (AI) to develop a NGS-COO classi ﬁ er for COO assignment and NGS survival models for clinical outcome prediction. The NGS-COO model built through applying AI in the training set was robust, showing high concordance with COO classi ﬁ cation by either Affymetrix GeneChip microarray or the NanoString Lymph2Cx assay in 2 validation sets. Although the NGS-COO model was not trained for clinical outcome, the activated B-cell – like compared with the germinal-center B-cell – like subtype had signi ﬁ cantly poorer survival. The NGS survival models strati ﬁ ed 30% high-risk patients in the validation set with poor survival as in the training set. These results demonstrate that targeted RNA-Seq coupled with AI deep learning techniques provides reproducible, e ﬃ cient, and affordable assays for clinical application. The clinical grade assays and NGS models integrating both genetic and transcriptional factors developed in this study may eventually support precision medicine in DLBCL. reproducible DLBCL-COO subclassification in daily clinical practice using a commercial available NGS platform; streamline analysis of high-throughput RNA-Seq data, COO assignment, and risk prediction by AI can further improve the workflow. Data portal and streamline AI integration can be further developed based on results in this study, which may provide a reliable tool for precision medicine and decision making by clinicians. However, the predictive and therapeutic values of the COO and risk stratification models developed in this large cohort of DLBCL remain to be determined in future prospective clinical studies.

• A DLBCL cell-of-origin classifier integrating both genetic and gene expression signatures shows robust predictive values in DLBCL cohorts.
• Targeted NGS and AI enable potential application of fast and reliable DLBCL classification assays in clinical practice.

Introduction
Diffuse large B-cell lymphoma (DLBCL) is the most common B-cell lymphoma and is clinically heterogeneous. Gene expression profiling (GEP) classified DLBCL into 2 major molecular subtypes according to their cell of origin (COO): germinal-center B-cell-like (GCB) and activated B-cell-like (ABC) DLBCL. 1 ABC-COO is associated with poorer clinical outcomes in DLBCL irrespective of treatment: CHOP (cyclophosphamide, doxorubicin, vincristine, and prednisone), rituximab (R)-CHOP, 1-3 obinutuzumab (G)-CHOP, 4 or classical salvage chemotherapy R-DHAP (rituximab, dexamethasone, high-dose cytarabine, and cisplatin) followed by intensive therapy plus autologous stem cell transplantation. 5 However, several novel agents, including lenalidomide, [6][7][8] ibrutinib, 8,9 and bortezomib alone 10 or in combination with durvalumab (anti-PD-L1), 11 showed selective or better clinical efficacy in ABC-vs GCB-DLBCL. The prognostic and therapeutic differences between ABC-and GCB-DLBCL have a molecular basis, such as higher frequencies of mutations in CD79, MYD88, CARD11, PRDM1, and TNFAIP3, 12 chronic active B-cell receptor signaling, 13 and more frequent MYC/BCL2 double expression in the absence of genetic MYC/BCL2 double hit 14 in ABC-DLBCL. In addition, the subcellular distribution and mechanism of action of doxorubicin in ABC-DLBCL are different from those in GCB-DLBCL. 15 To guide clinical therapeutics, distinction of the GCB vs ABC/non-GC subtype has become the standard practice according to the 2016 revision of the World Health Organization classification of lymphoid neoplasms. 16 Significant efforts have been put into establishing clinically applicable assays and accurate classification of DLBCL, and methodology to determine COO has been evolving in the last 2 decades. The original Lymphochip spotted cDNA microarray and the gold standard classification algorithm are robust in COO classification but impracticable for routine clinical practice. [1][2][3] Researchers thus developed algorithms to distinguish GC from non-GC subtypes based on protein expression of 3 to 5 biomarkers in formalin-fixed, paraffin-embedded (FFPE) tissue samples readily assessed by immunohistochemistry (IHC) in the clinic. [17][18][19][20][21][22][23][24] However, the accuracy of these IHC algorithms and the prognostic significance of COO subtypes determined by IHC algorithms 5,25 are not consistent. 23,[26][27][28] To enable GEP by DNA microarrays to classify DLBCL using clinical FFPE tissues that yield highly fragmented RNA samples, new RNA amplification and labeling techniques and classification models were developed, including a 100-gene classifier for Affymetrix GeneChip (Affymetrix, Inc) data 29 and a 20-gene DLBCL Automatic classifier for Illumina WG-DASL platform (Illumina United Kingdom) data 30 developed from a previous platform-independent 27-gene DLBCL subgroup predictor 31 that showed reproducibility and prognostic value.
To simplify the GEP process for FFPE samples, a multiplexed quantitative nuclease protection assay (qNPA) was developed that directly hybridizes mRNA in situ using 50-mer probes for genes of interest, followed by probe capture and quantitative imaging, thereby reliably detecting mRNA levels in FFPE samples without RNA exaction and amplification. [32][33][34] The qNPA platform (HTG Molecular Diagnostics, Inc.) can accurately classify DLBCL using a 14-gene signature. 35 The current HTG EdgeSeq DLBCL COO assay has been applied in a clinic trial. 36 However, the most successful simplified variation of microarray for rapid COO determination is the NanoString nCounter System (NanoString Technologies), which elegantly detects target mRNA of interest in extracted nonamplified RNA samples using a capture probe and a color-coded reporter probe, followed by purification, immobilization, and digital readout. 37 Several different  small gene panel-based DLBCL-COO assays, including the most  wildly used Lymph2Cx 20-gene assay, 38 have been applied in  research studies and clinical trials, 4,39-45 although a large gene  panel (145 genes) was also achievable for the NanoString nCounter system. 46 COO determined by Lymph2Cx 20-gene assay either exhibited high concordance with GEP-determined COO or showed significant prognostic value in 4 retrospective studies 47-50 and a clinical trial, 51 but not in 2 clinical trials 52 and 1 retrospective study. 53 Reverse transcriptase-multiplex ligation-dependent probe amplification, which ligates the left and right probes annealed to cDNA target sequences, permitting amplification of specific genes, 54 is another type of assay that has been applied for DLBCL-COO classification based on expression of 14 or 21 genes. 55,56 This method is sensitive and cost-effective without using a dedicated platform but has relatively poor dynamic range and is unable to include some COO-specific genes. 55 DLBCL outcome predictors that link GEP signatures directly to clinical outcome instead of COO have also been developed, 2,3,57,58 but the reproducibility between different studies was poor, and the predictive value for therapies other than the standard treatment is uncertain. In contrast, COO classification with underlying biology basis 9 also have predictive values for novel therapies, as demonstrated in phase 1/2 and 2/3 clinical trials. 6 consensus clustering algorithm to analyze high-content genetic data of 574 and 304 patients, respectively, and uncovered genetically distinct subtypes within or independent of COO subtypes, most of which demonstrated robust prognostic significance and potential therapeutic relevance. 61,62 However, the pathogenic driver roles of many mutations in signatures vary or have not been validated, 63,64 and how to accurately assign a genetic subtype to new individual patients at presentation in real time is less clear than the current COO classification. In a phase 3 GOYA study (NCT01287741), 43 approximation of EZB, BN2, N1, and MCD subtypes based on presence of subtype founder gene alterations in targeted next-generation sequencing (NGS) data of 465 genes did not find prognostic effect, whereas clusters (C) C2, C3, and C5 identified by applying NMF consensus clustering to the study cohort showed poorer prognosis compared with C0, C1, and C4 clusters. In another prospective study from the LNH03B LYSA (Lymphoma Study Association) clinical trials with targeted NGS of 34 key genes and genomic copy number variation analysis, none of the genetic subtypes identified by the GenClass algorithm or NMF consensus clustering showed prognostic significance. 65 The inconsistent prognostic values could result from the highly variable sequencing panels and NGS data quality in different studies, inaccurate subtyping, and the clinical heterogeneity within defined genetic subtypes underscored by phenotypic biologic (eg, MYC/BCL2 expression 66 ) heterogeneity arising from many other underlying mechanisms, for example, epigenetic deregulation and genetic alterations in noncoding regions. 67 In fact, in the cohort of Schmitz et al, MCD patients with MYD88/ CD79B double mutations had better survival compared with other MCD patients, 66 and the EZB subtype has been further divided into the unfavorable EZB-MYC 1 and favorable EZB-MYC ─ subtypes recently by a LymphGen algorithm. 68 A LymphGen webtool has been public accessible and able to assign genetic subtypes to patients if the input is from a cohort but not if from only 1 patient.
Based on these previous studies, we hypothesized that combined high-throughput genetic and gene expression signature analysis may improve the DLBCL classification for prognostic stratification and therapeutic implication. To be clinically applicable, fast and economical assays on FFPE samples that provide both genetic and expression data with low sample input are needed. We therefore implemented targeted RNA sequencing (RNA-Seq) of 1408 genes with NGS technology that simultaneously sequences and quantitates expressed mRNA molecules in a single assay. Artificial intelligence (AI) was implemented to build predictive models based on both genetic and gene expression data of a large number of DLBCL FFPE samples. The robustness of the predictive models was tested in validation cohorts supporting our hypothesis.

Patients and methods
Patients To further validate the COO model, 60 independent DLBCL samples were obtained and classified into ABC/GCB subtypes using the Lymph2Cx NanoString nCounter assay according to the manufacturer's instructions.

GEP analysis
Raw RNA-Seq and Affymetrix GEP data were preprocessed and normalized by robust multichip average using the R package (version 1.65.1). 72 Two-class unpaired significance analysis of microarrays were performed to identify significantly differentially expressed genes (DEGs) between the 2 groups. 73 Gene expression data were analyzed via CLUSTER software using the average linkage metric and then displayed by JAVA TREEVIEW (https:// www.java.com/en). 74

RNA library construction and sequencing
The Agencourt FormaPure Total 96-Prep Kit was used to extract both DNA and RNA from the same FFPE tissue lysates using an automated KingFisher Flex and protocols as recommended by each manufacturer. Samples were selectively enriched for 1408 cancerassociated genes using reagents provided in an Illumina TruSight RNA Pan-Cancer Panel. The cDNA was generated from the cleaved RNA fragments using random primers during the firstand second-strand synthesis. Then, sequencing adapters were ligated to the resulting double-stranded cDNA fragments. The coding regions of expressed genes were captured from this library using sequence-specific probes to create the final library. Sequencing was performed on an Illumina NextSeq 550 System platform. Ten million reads per sample in a single run was required. The read length was 2 3 150 bp. The sequencing depth was 103 to 17393, with a median of 413. An expression profile was generated from the sequencing coverage profile of each individual sample using Cufflinks. Expression levels were measured using fragments per kilobase of transcript per million and further normalized using the B-cell PAX5 RNA expression levels to adjust for variability in the percentage of DLBC cells in samples.
Alignment of sequencing data and variant calling were performed with the DRAGEN Somatic Pipeline (Illumina) using tumor-only analysis against the GRCh37 reference genome to identify 2 classes of mutations: single nucleotide variants and indels. Tumor samples were analyzed without a matching normal.

DLBCL COO classification and clinical risk prediction modeling
To build robust DLBCL classification models, we randomly selected 60% of cases to fit (train) the model and then validated using the remaining 40% (validation set). Sixty independent DLBCL samples classified by Nanostring Lymph2Cx assay were used as a second validation set.
First, univariate significance tests were used to screen the large number of variables. Normalized RNA expression data and mutation data were included as variants to build a classification model. For interpretability and simplicity, we divided the gene expression values into 4 or 10 equal parts using the quartiles (Q1, Q2, and Q3) and deciles and selected mutation data of 39 highly recurrent genes that had mutations in at least 10 patients. Fisher's exact test was used after discretizing RNA expressions using their quartiles, and 228 variables were statistically significant with P , .01. After adjusting for multiple hypothesis testing using Benjamini-Hochberg's method and setting the cutoff for false discovery rate (FDR) at 0.01, statistically significant variables were narrowed down to 129. Finally, setting the cutoff for FDR at 0.0001, 48 variables were selected with either small adjusted P values or high area under the receiver operating curve (AUC).
We selected 252 DLBCLs with high confidence COO assignment to develop risk stratification models directly correlating with survival. We randomly selected 60% (152) of subjects as the training set to fit the model and tested the performance in the remaining 40% (100) patients. Kaplan-Meier and Cox proportional hazards (CPH) analysis was used to identify variables with significant prognostic impact.
Multiple statistical approaches were tested for modeling performance, and models built through deep learning techniques 75,76 were most predictive and robust. We used autoencoders for nonlinear transformations of autoencoded features into 2-dimensional latent space. Logistic regression and CPH models were used for building the COO model and clinical risk models, respectively. A flowchart of the study is provided in supplemental Figure 1.

Results
Mutation spectrum, GCB/ABC association, and prognostic significance in DLBCL In 418 patients, RNA-Seq data fulfilled the quality control criteria (supplemental Table 1) and were further analyzed. In total, 2207 nonsynonymous mutations occurred in 598 genes in 412 patients.
In the study cohort, each patient had 0 to 30 genes harboring mutations (median, 4 mutated genes); 322 genes had mutations in at least 2 patients. Figure 1A shows the case distribution of mutations in 66 genes occurred in at least 6 patients. The corresponding mutation and clinical data are in the supplemental Data. The clinical features of the 418 patients are summarized in Table 1.
Comparisons between GEP-classified GCB (n 5 172) and ABC (n 5 160) cases showed largely similar results (top significant genes are shown in Figure 1B), except that the associations of NOTCH2 with ABC and DUSP2 mutations with GCB became insignificant, and additionally BIRC6 mutation and IRF4 mutation showed significant association with GCB and ABC subtype, respectively. In  addition, the 34 GEP-unclassified cases had higher frequencies of CREBBP and BRAF mutations than both GCB and ABC subtypes.
Mutation status of each gene was analyzed for prognostic significance. Table 2 lists frequently mutated genes with significant mutational effects on overall survival (OS) by univariate analysis.

Development and validation of the NGS-COO classification model
RNA-Seq gene expression (supplemental Data), gene fusion, and mutation data were used to develop a model for DLBCL-COO classification in the training set. Fisher's exact test and multiple hypothesis testing adjustment were used to identify RNA-Seq variables showing significant difference between GCB and ABC subtypes. DEGs between GEP-classified GCB and ABC subtypes are visualized in Figure 2A (FDR , 0.1, fold change $ 1.42). Finally, the top 48 variables (Table 3; Figure 2B) that were significantly differed between GCB and ABC subtypes with FDR , 0.0001 or features of the data. The values of these 2 neurons formed a lowdimensional (2) representation of the data; that is, it aggregated the 48 variables into 2 latent features. The top 7 contributing variables to the latent features were MYD88 mutation, EZH2 mutation, RASGRF1 expression, MYBL1 expression, S1PR2 expression, SSBP2 expression, and IRF4 expression. Based on the latent features, a logistic regression model was built for GCB/ABC classification (named as NGS-COO classifier). As shown in Figure 3A, the autoencoder transformed the high-dimensional data into a 2-dimensional space where the 2 subtypes were easily separable (linearly) roughly with a diagonal line from (21,21) to (1, 1).   BTLA  IL2RA  AFF3  CARD11  FOXP1  LYN  IKZF1  CREB3L2  ETV6  CD44  AHR  STAT3  CCND2  FAM46C  IRF4  PIM1  RASGRF1  TBL1XR1  LRRK2  PTPN2  LRP5  COL9A3  METTL7B  PAK6  IRS1  CD22  LYL1  EBF1  LMO2  LRMP  BCAS4  CCND3  LCK  LPP  PTK2B  KANK1  AUTS2  BCL6  MYBL1  SSBP2  S1PR2  PDGFD  PTK2  PIK3CG  PAG1  The NGS-COO classifier developed from the training set was then applied to the validation set. A probability of scoring was generated for each case. Approximately 30% of the cases had a score between 0.5 and 0.75, indicating low confidence for classification. For the remaining 70% with high confidence for assigning to 1 of the 2 subtypes (probability of 0.8 or higher), the ABC vs GCB classification showed sensitivity and specificity of 96% and 97% for classification in the validation set. The accuracy/concordance rate with previous GCB/ ABC classification was 95.6%. The corresponding AUC was 96.2%.
In the training and validation sets, in total, 216 cases were determined as the ABC subtype and 202 cases as the GCB subtype. Differential expression of the 46 genes constituting the NGS-COO classifier in the study cohort is visualized in Figure 2B.
The new GCB/ABC cases were also associated with 1319 significant DEGs with FDR , 0.0001 in GEP analysis using our previous Affymetrix GeneChip DNA microarray data ( Figure 2B) and multiple biomarkers characterized in previous studies by our Consortium program (supplemental Table 2).
To further evaluate the performance of the NGS-COO classification model, we applied the same approach to 60 independent cases as an external validation cohort. Our NGS-COO model showed sensitivity and specificity of 96% and 97%, respectively, with the previous COO classification by the NanoString Lymph2Cx assay. The concordance rate was 92.9%. The corresponding AUC was 95.7%. As shown in Figure 3B-C, the pattern of separation by the autoencoder was similar between training, validation, and independent sets. They all showed separation between ABC and GCB with a diagonal line from (21,21) to (1,1), although the independent 60 cases were collected from a completely different set of samples and the sequencing was performed separately.
The performance of our NGS-COO classifier was also evaluated by correlating with survival outcomes. Although the autoencoder was only trained for COO classification in the training set, the NGS-COO classifier was significantly associated with OS and progression-free survival (PFS) in DLBCL, similar to the previous COO classification ( Figure 4A-C). The relative risk of the new ABC compared with the new GCB group was 1.53. The prognostic significance was slightly improved if comparing within highconfidence cases only (risk for OS, 1.81, P 5 .007; risk for PFS, 1.77, P 5 .0046).

Development of prognostic models for DLBCL risk stratification
To build robust prognostic models aggregating small contributions of a large number of variables directly to patient survival, we used a similar procedure and the AI method to develop models in the training set and test the performance in the validation set based on both gene expression and genetic variables plus 2 additional factors: age and sex of patients. We first screened for significant variables using Kaplan-Meier and CPH for OS in the training set. Although 61 variables showed significant prognostic effects by logrank test and 110 variables by CPH regression (P , .05), only the TP53 mutation remained statistically significant after adjusting for multiple hypothesis testing (FDR , 0.0001). Therefore, we selected 57 variables with the top 2% AUC values or P , .01 (either based on log-rank test or CPH; Table 4).
We used a similar neural network architecture as described for COO modeling and again included 2 neurons in the bottleneck layer to reduce the data into 2 dimensions (latent features). The top 7 variables contributing to the 2 latent features are age .60, TP53 mutation, CARD11 expression, BCL6 expression, MALAT1 expression, RABEP1 expression, and BCORL1 expression. A simple CPH model was built based on the 2 latent features obtained from the autoencoder (which are nonlinear combinations of the 57 variables) and provided a risk score (NGS-OS score) for each case,  which was normalized to be between 0 (lowest risk) and 100 (highest risk). As shown in Figure 5A, we divided the training set into 3 equal subgroups based on the NGS-OS risk score and found the high-risk group had strikingly poorer survival than the low-and intermediate-risk groups (P , .0001). We then applied the NGS-OS model into the validation set and stratified patients using the same NGS-OS risk score cutoffs established in the training set and found that 3 resulting risk groups in the validation set showed incremental survival rates. The relative OS risk for the 3 subgroups was roughly 1, 4, and 9.
We followed a similar procedure to build a CPH model for PFS with 50 selected variables based on a 2-dimensional feature set obtained from an autoencoder ( Table 5) Similar with the NGS-OS risk scores, NGS-PFS risk scores identified one third of the training set and 30% of the validation set as high-risk patients ( Figure 5B). The relative risk for the low-, intermediate-, and high-risk groups in the validation sets was roughly 1, 2, and 4, respectively.

Discussion
In this study, we developed novel DLBCL classification models based on both genetic and transcriptional variables derived from comprehensive RNA-Seq annotation and quantitative data. Molecular classification methods based on tumor biology can be categorized according to whether the model is based on normal or abnormal signatures and whether the measure is at the DNA (genetic), RNA (transcriptional), or protein level (summarized in the visual abstract according to the literature [1][2][3]14,[17][18][19][20][21][22][23][24]57,58,61,62,68,77,78 80 and AI is emerging as an efficient tool in health care for large data processing and sophisticate model construction, 76,81 currently no NGS panels and AI implementation have been developed for lymphoma diagnosis and management.
Our study supports the reliability and practicality of using targeted NGS along with AI in generating clinically useful objective information.
Compared with current IHC assays, DNA microarrays, and other GEP analysis techniques used for DLBCL COO classification, targeted RNA-Seq has a balanced advantage of genome-wide coverage, dynamic range of quantification, reproducibility, high throughput, and accuracy, as well as high sensitivity, automation, affordability, short assay time, and flexibility. 82 As RNA-seq has become less costly and been integrated into clinical practice, 80 we expect that the generated RNA-seq data will be used not only to answer the COO and prognostic questions but also other diagnostic and clinical questions impacting clinical decisions, such as predicting clinical responses to novel therapies in clinic and in future prospective or retrospective studies. 80,83 In conclusion, the current proof-of-principle study demonstrates the potential utility of the targeted RNA-Seq assay for accurate and  reproducible DLBCL-COO subclassification in daily clinical practice using a commercial available NGS platform; streamline analysis of high-throughput RNA-Seq data, COO assignment, and risk prediction by AI can further improve the workflow. Data portal and streamline AI integration can be further developed based on results in this study, which may provide a reliable tool for precision medicine and decision making by clinicians. However, the predictive and therapeutic values of the COO and risk stratification models developed in this large cohort of DLBCL remain to be determined in future prospective clinical studies.