{'fraction_non_alphanumeric': 0.033398622180197214, 'fraction_numerical': 0.02046467648250709, 'mean_word_length': 5.334331550802139, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 2, 'https://': 0, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Evaluation of human epidermal growth factor receptor 2 status of breast cancer using preoperative multidetector computed tomography with deep learning and handcrafted radiomics features

Objective To evaluate the human epidermal growth factor receptor 2 (HER2) status in patients with breast cancer using multidetector computed tomography (MDCT)-based handcrafted and deep radiomics features. Methods This retrospective study enrolled 339 female patients (primary cohort, n=177; validation cohort, n=162) with pathologically confirmed invasive breast cancer. Handcrafted and deep radiomics features were extracted from the MDCT images during the arterial phase. After the feature selection procedures, handcrafted and deep radiomics signatures and the combined model were built using multivariate logistic regression analysis. Performance was assessed by measures of discrimination, calibration, and clinical usefulness in the primary cohort and validated in the validation cohort. Results The handcrafted radiomics signature had a discriminative ability with a C-index of 0.739 [95% confidence interval (95% CI): 0.661−0.818] in the primary cohort and 0.695 (95% CI: 0.609−0.781) in the validation cohort. The deep radiomics signature also had a discriminative ability with a C-index of 0.760 (95% CI: 0.690−0.831) in the primary cohort and 0.777 (95% CI: 0.696−0.857) in the validation cohort. The combined model, which incorporated both the handcrafted and deep radiomics signatures, showed good discriminative ability with a C-index of 0.829 (95% CI: 0.767−0.890) in the primary cohort and 0.809 (95% CI: 0.740−0.879) in the validation cohort. Conclusions Handcrafted and deep radiomics features from MDCT images were associated with HER2 status in patients with breast cancer. Thus, these features could provide complementary aid for the radiological evaluation of HER2 status in breast cancer.


Introduction
Breast cancer is the most common malignancy in women worldwide and remains the first cause of female cancerrelated death (1,2). Breast cancer is a heterogeneous disease with diverse phenotypes that exhibit distinctive biological behaviors (3). Approximately 15%−20% of breast cancers manifest as human epidermal growth factor receptor 2 (HER2) protein overexpression or gene amplification, which is defined as an aggressive subtype associated with metastatic behavior and poor clinical outcomes (4). Nevertheless, the prognosis of the HER2-positive subtype of breast cancer has substantially improved since the development of anti-HER2 targeted therapies. Female patients with HER2-positive breast cancer show good responses and high pathological complete response rates after neoadjuvant chemotherapy with the HER2-blockade agent trastuzumab; further, these patients demonstrate a considerable improvement in both disease-free and overall survival (5-7). Therefore, HER2 status is important for the prognosis of breast cancer and in choosing the optimal individualized treatment strategy for such patients.
HER2-positive cancers tend to accelerate the growth and division of cancer cells, as well as stimulate the cell proliferation and angiogenesis, all of which may cause the tumor heterogeneity (8). In recent years, studies have suggested that the biological characteristics of tumors could be captured using medical images at both genetic and cellular levels (9). Radiomics, the extraction and analysis of quantitative imaging features, enables imaging phenotypes to be correlated with genetic information, which is of great significance for diagnosis, choosing individualized treatment strategies, and predicting the prognosis of tumors (9)(10)(11)(12)(13).
Recent studies have tried to use imaging features to assess their associations with the HER2-positive subtype of breast cancer. Although breast radiomics features derived from magnetic resonance imaging (MRI) (14)(15)(16) and mammography (MG) (17) are reportedly associated with the HER2-positive subtype, other studies have failed to find a correlation between HER2 status and the features extracted from MRI (18) and positron emission tomography/computed tomography (PET/CT) (19). Accordingly, the current research is still insufficient due to the conflicting relevance; thus, further exploration is required. Multidetector computed tomography (MDCT) also plays an important role in the clinical practice of breast cancer (20), and the 2019 National Comprehensive Cancer Network (NCCN) guidelines of Breast Cancer (version 3. 2019) recommend chest contrast CT examination for patients with breast cancer if pulmonary symptoms are present. Moreover, many patients with breast cancer would undergo MDCT for other reasons as well (e.g., chest pain). The radiomics features within MDCT images may be correlated with the HER2 status in breast cancer, which could provide supplementary assistance with non-invasive imaging evaluation.
Furthermore, conventional radiomics studies generally extracted handcrafted features, which quantify tumor shape, intensity, and texture information based on imaging. However, low-order handcrafted features may be inadequate as they reveal information about medical images from limited aspects; therefore, the tumor heterogeneity may not be fully characterized. Recently, with developments in image recognition and analysis tools, deep learning has drawn increased interest. Deep learning technology, especially convolutional neural network (CNN), is an artificial intelligence algorithm that learns on its own to extract the most predictive features directly from pixel images (21) and has shown remarkable classification and recognition performances in image analysis (22). Compared to handcrafted features, deep features reflect information in medical images from a different perspective and may add further predictive value to HER2 status prediction (23).
To the best of our knowledge, no studies have investigated the correlations between the radiomics features of MDCT images and HER2 status in patients with breast cancer. Therefore, the purpose of this study was to examine if the combination of handcrafted radiomics features and deep learning features based on preoperative MDCT images could evaluate the HER2 status of breast cancer and thus provide complementary aid for the radiological evaluation of HER2 status in patients with breast cancer.

Patient population
This retrospective study was approved by the Medical Ethics Committee of Guangdong Provincial People's Hospital, Guangdong Academy of Medical Sciences, and the requirement for informed consent was waived. The inclusion criteria were as follows: 1) patients who underwent preoperative contrast-enhanced MDCT between January 2016 and December 2018 with visible breast cancer on the images; 2) histopathological verification of primary invasive breast cancer with surgical resection; and 3) HER2 expression status confirmed by immunohistochemistry (IHC) and fluorescent in situ hybridization (FISH) tests of the surgical specimen. The exclusion criteria were as follows: 1) patients with incomplete clinicopathologic data; 2) patients who were treated with neoadjuvant chemotherapy before surgery; or 3) equivocal HER2 status determined by IHC and FISH tests. The patient recruitment flowchart is presented in Supplementary Figure S1.
In total, 339 female patients who met the criteria were included in this study. Of these, 27 patients had multi-focal or multi-centric lesions, and only the largest tumor lesion of each patient was used for analysis. Finally, the 339 patients were randomly divided into the primary cohort (n=177; age, 50.53±10.49 years old; range, 27−78 years) and the independent validation cohort (n=162; age, 51.88±8.71 years old; range, 35−73 years). The baseline clinical information, including age and tumor location, of recruited patients was collected from the institution archives.

Assessment of HER2 status
The HER2 status of all breast cancer patients included in this study was detected using the surgical specimen without neoadjuvant chemotherapy, and determined using IHC or FISH tests according to the 2013 guideline recommendations of the American Society of Clinical Oncology and College of American Pathologists. The IHC staining intensity of HER2 was graded as 0, 1+, 2+, or 3+. Grades 0 and 1+ were defined as negative, whereas grade 3+ was considered positive. Grade 2+ was equivocal and was further confirmed using FISH, which considered a HER2 gene copy number ≥6 or a HER2/chromosome enumeration probe 17 (CEP17) ratio ≥2.0 as confirmation of HER2 protein overexpression (24).

Radiomics model workflow
The workflow of radiomics features modelling is illustrated in Figure 1 and includes tumor segmentation and the resized process; handcrafted features and deep features extraction; feature selection; and radiomics signatures and model construction.

MDCT image acquisition and segmentation
Preoperatively, all patients underwent contrast-enhanced chest CT scans, which was performed at different MDCT facilities at Guangdong Provincial People's Hospital between January 2016 and December 2018. A more detailed description regarding image acquisition and segmentation is provided in Supplementary materials.

Feature extraction
Handcrafted feature extraction In this study, four categories of handcrafted radiomics features were extracted for analysis: 1) first-order statistics features; 2) size-and shape-based features; 3) texture features; and 4) filter features. The extraction of handcrafted features is described in detail in the Supplementary materials. Handcrafted feature generation was performed via a toolbox developed in-house using MATLAB 2016b (Mathworks, Natick, MA, USA).

Deep feature extraction
Since medical images typically have a limited dataset compared to natural image sets, causing difficulties in training CNN models from scratch, the transfer learning has been proposed to overcome this shortage. Transfer learning is an approach which uses pre-trained models from images of other domains and makes them useful for new datasets (25). Currently, transfer learning is widely implemented in the area of medical deep learning and may alleviate the limitation of small datasets (26).
In this study, transfer learning was performed to extract deep features from two-dimensional MDCT images. Convolutional neural networks fast (CNN-F) (27), as the CNN model used in our study, consisted of five convolutional layers and three fully connected layers, and was pre-trained using the ILSVRC-2012 dataset. The hyperparameters of the pre-trained model were the same as those used by Krizhevsky (28) and are presented in Supplementary materials.
For each patient, the section containing the largest tumor area was selected as the input into the CNN-F. The pre-trained model required three-channel input images (RGB-coded images); however, the medical images in DICOM format were single-channel gray images. Therefore, we first selected and manually segmented the largest tumor area section and cropped the tumor area. Then, the gray values of the segmented region of interest (ROI) were converted into the range (0, 255) using linear transformation (29) and each image was rescaled to 224×224 pixels using bicubic interpolation. Next, three rescaled images were used as the R, G and B channels and stacked into a three-channel image (224×224×3), which met the requirement of the pre-trained CNN-F model input. Finally, the deep features were calculated by forward propagation, and the features of the fully connected layer prior to last fully connected layer were extracted as deep features for subsequent analysis.

Feature selection
To construct effective and robust radiomic signatures, a coarse to fine strategy was employed for feature selection. Firstly, depending on the different combinations of the independent segmentation of 100 patients, intra-and interclass correlation coefficients (ICCs) were used to determine the robust features. Features with ICC values >0.75 were classified as robust features for further analysis. Secondly, univariate analysis (the Mann-Whitney U test) was performed to compare the robust radiomics features between HER2-positive and HER2-negative groups. All features were sorted in ascending order in terms of P values generated from the univariate analysis; of the features, the top 20% were selected. Thirdly, the support vector machine with recursive feature elimination (SVM-RFE) algorithm was used for further feature selection (30). SVM-RFE is an efficient feature selection algorithm which ranks the features according to the weight of features based on support vectors. Finally, the number of key features selected for building a radiomics signature was determined by the C-index value using 10-fold cross-validation. This procedure was implemented on both handcrafted feature and deep feature selections in the primary cohort.

Radiomics signature construction
Using the selected key handcrafted features and deep features, the handcrafted-and deep-radiomics signatures for predicting HER2 status were respectively developed using multivariate logistic regression in the primary cohort. We calculated the handcrafted radiomics score (HRadscore) and deep radiomics score (DRad-score) for each patient with a linear combination of the selected features which were respectively weighted by their normalized coefficients. The signatures trained on the primary cohort were applied to the validation cohort for testing in independent cases.

Prediction model development
Multivariate logistic regression was used to select clinical predictive factors (i.e., age, tumor location) for developing the prediction model in the primary cohort. Handcrafted-and deep-radiomics signatures were applied to develop a prediction model in the primary cohort.

Statistical analysis
The differences in clinical characteristics and radiomics scores between the HER2-positive and HER2-negative patients in both the primary and validation cohorts were assessed. The two-sided Mann-Whitney U test was used for continuous variables (i.e., age, Rad-score) and the Chisquare test for categorical variables (i.e., tumor location). The differences in predictive performance between the combined model and the radiomics signatures were tested using the Delong test. Statistical analysis in this study was performed with R software (Version 3.5.2; http://www.Rproject.org). The R packages that were used in this study are listed in the Supplementary materials. P-values <0.05 were considered to be statistically significant.

Evaluation of signatures and model performance
To evaluate the radiomics signatures and prediction model performance in this study, we measured the overall performance, discrimination, calibration and clinical usefulness of the model in the primary cohort and then validated it in the validation cohort (31).

Overall performance
Brier score (32) was calculated to assess the overall performance of the radiomics signatures and the prediction model. The Brier score provided a measure of the agreement between the observed binary outcome (i.e., HER2 positive vs. HER2 negative) and the predicted probability of that outcome. The Brier score for a model can range from 0 (perfect model) to 0.25 (non-information model). Generally, a lower Brier score implies better model calibration and discrimination.

Discrimination
The C-index was used to measure the discriminative ability of the radiomics signatures and prediction model. It is equal to the area under the receiver operating characteristic (ROC) curve and varies from 0.5 (no apparent accuracy) to 1.0 (perfect accuracy) (33).

Calibration
Calibration was used to describe the consistency between the actual outcomes and the predictions. The calibration curve was graphically presented as an assessment of calibration, with predictions on the x-axis and the actual outcome on the y-axis. Perfect predictions should be on the 45-degree line. The Hosmer-Lemeshow test was applied to assess the goodness-of-fit of the model, and a high P-value (>0.05) is considered to be reasonable calibration.

Clinical usefulness
To evaluate the clinical utility of the combined model, the decision curve analysis was applied (34). In this study the standardized net benefit (sNB), which ranges from 0 to 1, was used as a function of the risk threshold in the decision curve with visualization. The clinical impact plot was used to visually show the estimated number of patients that had been deemed as high risk for each risk threshold and the true positive cases. An ROC components plot was used to show the constituents of sNB (i.e., the true and false positive rates) (34).

Patients' clinical characteristics
There was no significant difference in the HER2 status between the primary and validation cohorts (P=0.893). There were also no significant differences in clinical characteristics between the two cohorts (P=0.198−0.893).
The differences in clinical characteristics between the HER2-positive and HER2-negative groups in both the primary and validation cohorts are shown in Table 1. No significant differences were found in clinical characteristics (age and tumor location) between the HER2-positive and HER2-negative patients in either cohort (P=0.065−0.883).

Feature selection and radiomics signature construction
In total, 5,013 handcrafted and 4,096 deep features were extracted from the ROIs of patients with breast cancer. After feature selection, 7 handcrafted and 7 deep features with preferable predictive value were finally selected in the primary cohort to construct the handcrafted and deep radiomics signatures, respectively. The handcrafted radiomics (HRad)-and deep radiomics (DRad)-scores of each patient were calculated using the formula constructed by the respective selected features (Supplementary materials).

Evaluation of handcrafted and deep radiomics signatures
The handcrafted radiomics signature demonstrated  Table 2.

Performance of combined model
No significant clinical predictors of HER2 status in breast cancer were found using logistic regression analysis, whereas the handcrafted and deep radiomics signatures were identified as independent predictors. Therefore, a combination of the handcrafted and deep radiomics signatures was used to form the prediction model in this study.
The combined model showed better performance for the prediction of HER2 status in breast cancer than either of the two radiomics signatures in both cohorts ( Table 2). The overall performance of the combined model was improved; the Brier scores decreased from 0.220 and 0.231 to 0.211. The combined model showed good discriminative ability, achieving a C-index of 0.829 (95% CI: 0.767−0.890) in the primary cohort and 0.809 (95% CI: 0.740−0.879) in the validation cohort.
The calibration curve of the combined model for the prediction of HER2 status demonstrated good agreement between the predicted and observed outcomes in both the primary and validation cohorts ( Figure 2). The Hosmer-Lemeshow test was non-significant in the primary (P=0.887) and validation cohorts (P=0.528), indicating goodness of fitness.
The decision curve analysis for the two radiomics HRad-score, median (IQR) HER2, human epidermal growth factor receptor 2; HRad-score, handcrafted radiomics score; DRad-score, deep radiomics score; IQR, interquartile range; P value is calculated from the univariable association analyses between each of the clinicopathological variables and HER2 status. signatures and the combined model in both cohorts is presented in Figure 3A,B. The decision curve analysis plot showed that the combined model outperformed the two radiomics signatures in the range of thresholds from 0.2 to 0.8. The clinical impact plot showed that, if a 0.5 risk threshold was used, the number of cases identified as high risk of expressing HER2-positive was close to the number of true HER2-positive cases ( Figure 3C,D). Finally, the true and false positive rates were displayed as functions of the risk threshold in the ROC components plot ( Figure 3E,F).

Discussion
In this study, we explored the potential association between the HER2 status of breast cancer and radiomics features extracted from MDCT images. The combined model, which incorporated both handcrafted and deep radiomics features, showed a good performance in evaluating HER2 status; thus, the combined model may be useful as a complementary aid in the radiological evaluation of breast cancer. HER2-positive status has been shown to be a poor prognostic indicator for breast cancer, which also simultaneously means significant benefits from anti-HER2 targeted therapies (7). Accurately correlating radiological features with the HER2 status in the radiological evaluation of patients with breast cancer is important. Although MDCT is not the primary radiological method for breast cancer evaluation, at Guangdong Provincial People's Hospital, most patients with breast cancer undergo routine MDCT scans for staging before surgery or neoadjuvant chemotherapy. MDCT is also used for the follow-up of patients with advanced breast cancer. MDCT is advantageous as it has a fast scan time and is capable of multi-planar reconstruction; further, it can be performed when patient are in the supine position (35). It can also simultaneously evaluate the extension of lesions, as well as region lymph nodes, the skin, chest wall and distant metastasis (20).
A previous study by Tamaki et al. found that MDCT features (tumor shape, enhancement pattern and density) are distinct among breast cancer subtypes (36). In recent years, radiomics has appeared as an emerging field that converts medical imaging into quantitative features, which may improve clinical diagnosis, prognosis and prediction, especially in oncology (12,37). In view of the ability of radiomics to noninvasively decode tumor heterogeneity, current studies of other tumors have shown a potential correlation between tumor genotype and CT-based radiomics characteristics (11,38,39). In this study, we examined if radiomics features based on MDCT images reflect HER2 status in patients with breast cancer.
Many previous studies have tried to investigate the relationship between image characteristics and the HER2enriched subtype in breast cancer; the most commonly used imaging technique was MRI (14,15), followed by MG (17) and PET/CT (19). Chang et al. have demonstrated that breast MRI features that quantify the tumor heterogeneity could be used to indicate HER2 status with an area under the curve (AUC) of 0.8458 (16). Although the AUC was higher than that of our study, it lacked independent validation. Moreover, other studies have found no correlation between the HER2 status and imaging features extracted from MRI (18) and PET/CT (19). As for mammographic features (including tumor size, nonspiculated mass, and calcification) in the study of Nie et al. (40) and MG-based quantitative radiomics features from study of Ma et al. (17) and Zhou et al. (41), above studies reported good performances in differentiating the HER2enriched subtype, with AUCs of 0.75, 0.78 and 0.787, respectively. The above predictive performances were both slightly poorer than the C-index (which is equal to the AUC) of our study in the validation cohort (C-index: 0.809). Additionally, compared with the generally small HER2-positive dataset in most of the above-mentioned studies, our study enrolled 117 HER2-positive patients (117/339) with surgical pathology for analysis. Therefore, the performance of MDCT radiomics features provides a complementary aid for the radiological evaluation of HER2 status in breast cancer.
In recent years, the integration of multiple markers into one model has been shown to be advantageous for the individualized management of patients and has also been shown to outperform the use of individual markers (12,42). To improve the performance of imaging features in our study, we extracted and incorporated handcrafted and deep radiomics features into the predictive model. CNNs are deep learning models which learn increasingly higher level features from input images through a series of successive linear and nonlinear layers (21,43). Compared to conventional handcrafted radiomics features, high-order deep radiomics features could provide further supplementary information to elevate the performance of the model (44,45). In this study, the model that incorporated both handcrafted and deep radiomics signatures outperformed the individual radiomics signatures in HER2 status evaluation, with a good discriminative ability (C-index=0.829 in the primary cohort; C-index=0.809 in the validation cohort).
As a preliminary study, this study had some limitations. First, we examined a relatively small dataset from a single institution. Second, we only used two-dimensional radiomics features of the largest slice for analysis. Third, MDCT is only a single radiological method; further combinations with radiomics findings from multiple radiological methods including MR and MG are expected to improve the radiological evaluation for HER2 status.

Conclusions
This study indicated that handcrafted and deep radiomics features extracted from MDCT images are associated with HER2 status in breast cancer and may provide complementary, noninvasive assistance in the radiological evaluation of the HER2 status in patients with breast cancer.

26.
Chatfield K, Simonyan K, Vedaldi A, et al. Return of the devil in the details: Delving deep into convolutional nets. Computer Science 2014.

28.
Sasikala V, LakshmiPrabha V. A comparative study on the swarm intelligence based feature selection 29.

Image acquisition and segmentation
Acquisition parameters of multidetector computed tomography (MDCT) scanners were listed in Supplementary Table S1. After all CT images of each patient were retrieved from the picture archiving and communication system (Carestream, Canada) in Guangdong Provincial People's Hospital, we chose arterial phase for image segmentation and analysis, because this is the phase when the enhancement of tumor lesion showed favorable distinction from adjacent normal glandular tissues.
For handcrafted and deep features extraction, tumor region of interest (ROI) was segmented in the largest cross-sectional area of the CT image using the free open source software ITK-SNAP (Version 3.6.0, http://www.itksnap.org). All ROIs were manually delineated along the tumor outline with exclusion of the air and necrosis area that caused by biopsy. To evaluate the inter-observer and intra-observer reproducibility of the feature extraction, 100 cases were randomly selected for ROI segmentation and feature extraction. The ROI segmentation was performed by two radiologists, Reader 1 and Reader 2, with 8 and 10 years of chest CT interpretation experience, respectively. Reader 1 then repeated the process one month later. Intra-and inter-class correlation coefficients (ICCs) were applied for the evaluation of the intra-and inter-observer agreement of feature extraction. The remaining image segmentation was performed by Reader 1 who was blinded to the patient's clinicopathological information.

Handcrafted feature extraction
In view of CT images were collected from different CT scanners, we standardized the images to minimize bias and achieve predictive consistency. The ROIs of segmented images were resampled to 1 mm × 1 mm pixels before feature extraction by using bicubic interpolation.

Size-and shape-based features
Size-and shape-based features mainly reflects the morphological features of the tumor.

First-order statistics features
First-order statistics features were mainly used to describe the distribution of pixel intensities in CT images.

Texture features
Texture features mainly describe the spatial relationship between pixels. In this study, gray level co-occurrence matrix (GLCM), gray level run length matrix (GLRLM), gray level size zone matrix (GLSZM.), gray level dependence matrix (GLDM), and neighborhood gray tone difference matrix (NGTDM) were used to describe the feature of images.

Deep learning feature extraction
In this study, transfer learning was performed to overcome the challenge of the limited amount of data for deep learning. Convolutional neural network (CNN)-F, as the CNN model used in our study, consists of five convolutional layers and three fully connected layers and is pre-trained on the ILSVRC-2012 dataset. The hyper-parameter of the model is the same as that used by Krizhevsky: weight decay 5 × 10 −4 , momentum 0.9, initial learning rate 10 −2 . When the validation error stops reducing, the initial rate drops to 0.1.

V i 2 [1; 4096]
A total of 4,096 features was extracted from the last fully-connected layer. These features have a very abstract meaning, and we use a special nomenclature to name them as .

Statistical analysis
The following R packages were used in the study: Caret: for feature selection rms: logistic model building, nomograms, calibration plots, VIF rmda: decision curve analysis, clinical impact plot ResourceSelection: Hosmer-Lemeshow goodness of fit test

Calculation of radiomics score
Handcrafted radiomics signature After feature selection produce, 7 handcrafted features were selected. A handcrafted radiomics signature was constructed with 7 handcrafted features by using the logistic regression model. The handcrafted radiomics score for each patient was computed by follow formula (the P value of coefficients and abbreviation description of 7 handcrafted radiomics features were show in Supplementary Table S2):