{'fraction_non_alphanumeric': 0.04545902770929888, 'fraction_numerical': 0.029188442954343752, 'mean_word_length': 5.120398310199155, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
The value of S-Detect for the differential diagnosis of breast masses on ultrasound: a systematic review and pooled meta-analysis

1Department of Medical Ultrasound, the First Affiliated Hospital of Medical College, Shihezi University, Shihezi, China, 2Department of Medical Ultrasound, Wuchang Hospital affiliated Wuhan University of Science and Technology, Wuhan, China, 3Department of Medical Ultrasound, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan China, 4Department of Ultrasound, The First People’s Hospital of Huaihua, University of South China, Huaihua, China, 5Department Allgemeine Innere Medizin (DAIM), Kliniken Hirslanden Beau Site, Salem und Permancence, Bern, Switzerland


Introduction
Breast cancer remains a significant scientific, clinical and societal challenge [1,2]. It is the second most common cancer in the world and the most frequently diagnosed cancer in women [3][4][5]. Approximately 11% of worldwide breast cancers occur in China and the incidence has increased rapidly in recent decades [6]. Some studies have shown that the age, geographic locations, family history, hormone and reproductive history (including nulliparity, late age at first childbirth, early age at menarche and use of oral contraceptives or hormone replacement therapy) are related to the incidence of breast cancer [6][7][8]. But its pathogenesis is complex and the high-risk factors are difficult to control. The early accurate diagnosis and control of breast cancer are important for the prognosis [9,10].
Breast ultrasound (US) is often used as an adjunct to mammography and magnetic resonance imaging (MRI) to characterize focal breast lesions, thereby improving cancer detection rates [11]. However, US is an operator-dependent imaging modality that requires experience to depict the breast lesion and diagnose it, in which the reproducibility of the images should also be considered.
Many new imaging techniques have been developed such as ultrasound elastography, contrast-enhanced ultrasound (CEUS), three-dimensional US and automatic breast full-volume scanning imaging system (ABVS) which all have provided more convenience [12,13].
In the recent years, artificial intelligence (AI), from machine learning (ML) to deep learning (DL) algorithms, is gaining extensive attention for its excellent performance in image-recognition tasks [9,[14][15][16]. S-Detect (Samsung Medison Co. Ltd., Seoul, Korea) is a recently developed ultrasound computer assistant diagnosis (CAD) system using deep learning, and has become one of the most increasingly used CAD systems for the diagnosis of breast cancer. It can implement auto-segmentation and interpretation basically of US morphological descriptions, allowing classification of breast lesions in a dichotomic form (possibly benign or possibly malignant) as a reference for radiologists to assist with the final diagnosis. But it is not automatic segmentation in the full sense and needs to be adjusted manually a little if the outline is not good [17]. It is known as a clinically feasible diagnostic tool with a moderate degree of agreement in the final assessment, regardless of the experience of the radiologists specializing in breast imaging [18]. But the diagnostic criteria of the S-Detect classification system are not clearly defined in terms of whether the lesion is calcified, the distribution of blood vessels, changes in surrounding tissues, and the presence of enlarged lymph nodes. It still requires artificial judgment and increases subjectivity.
As a new technique, the application of S-Detect in clinical practice is still controversial and the results vary from different clinical trials. In the results from Jiang et al, the sensitivity was 100% [13], while Gewefel et al showed a sensitivity of just 61.90% [19].
At present, the sensitivity of the results varies greatly among studies and there is no meta-analysis and guidance on this technique for the diagnosis of breast cancer. Therefore, we conducted a meta-analysis to assess the performance of S-Detect in discriminating benign and malignant breast masses.

Search strategy
Literature retrieval was performed to select all relevant studies published before 15th July 2019 from databases including PubMed, Embase, China National Knowledge Infrastructure (CNKI), China Biology Medicine disc, WANFANG databases, Cochrane Library, Google Scholar and Web of Science. The terms used included "computer aided diagnosis", "breast nodules", "breast masses", "breast cancer", "breast neoplasm", "breast malignant neoplasm", "breast tumor", "diagnosis", "diagnostic" "S-Detect", "smart detect", "ultrasound", and "artificial intelligence". The searches were limited to identify the diagnostic studies without language restrictions.

Study selection
After searching the candidate studies for inclusion, criteria were set for further identification. The remaining studies were included in the meta-analysis if the following inclusion criteria were satisfied: 1) Studies evaluating the diagnostic accuracy of S-Detect; 2) The diagnosis of malignant and benign breast masses should be confirmed by histopathological examination; 3) Diagnostic accuracy statistics obtained by using this technique alone, including true-positive (TP), false-positive (FP), true-negative (TN), false-negative (FN), should be provided or can be calculated; 4) Clinical studies should have received informed consent from each study subject and obtained approval from the local ethics committees. Furthermore, studies were excluded if: 1) Studies did not provide sufficient data for calculating the TP, FP, TN, and FN parameters or only had obtained t results of the combined diagnosis of the technique and the doctor; 2) Studies incorporated duplicated data; 3) Studies that were not approved by local ethics committees or did not acquire informed consents from all the study subjects. Relevant studies were selected by two researchers independently based on title and abstract or full article. Any discrepancies concerning the study selection were resolved by discussion of the full article.
Data extraction and quality assessment The following information were extracted using a standardized form: the first author's surname, year of publication, country of the study, number of masses available for analysis, reference standard, mean age of patients, mean size of masses, type of the US system, TP, FN, FP, and TN numbers. The quality of included studies was assessed using the Quality Assessment of Diagnostic Accuracy Studies (QUADS) questionnaire by the same two observers [20]. The Revman 5.3, special software for Cochrane collaborative network, was used to output the result of QUADS.

Statistical analysis
After data extraction, the pooled sensitivity (PSEN), pooled specificity (PSPE), positive likelihood ratio (PLR) and the negative likelihood ratio (NLR) were estimated by the bivariate model and the summary receiver operating characteristic (SROC) [21]. The post-test probabilities were calculated by the PLR and NLR and plotted on a Fagan nomogram. The SROC curve with 95% confidence and prediction regions was also plotted to illustrate the relationship between sensitivity and specificity. In addition, the publication bias was assessed using the Deeks' method. All the data analysis and the graphs were made using the STATA version 14.0 software for Windows (Stata Corp, College Station, TX) with the commands MIDAS and METANDI. p<0.05 was regarded as statistically significant. RevMan software, version 5.3 (The Nordic Cochrane Center, The Cochrane Collaboration, Copenhagen, Denmark) was used to make flowcharts and assess the quality of the research included. Meta-Disc software, Version 1.4 (Madrid, Spain) was used to test the threshold effect. And the meta-regression analyses were used to deal with the existence of heterogeneity.

Literature searches
Based on the key words, 219 related papers were initially found. After screening the title and abstract, 38 articles were filtered automatically as duplicates, another 167 articles were removed because they were unrelated to the topic or belonged to review. From 14 full-text articles assessed for eligibility, 4 papers were further excluded because of insufficient data or for other reasons. Finally, 10 studies were included in the meta-analysis (fig 1).

Methodology quality assessment
According to the methodological assessment by the QUADAS-2 checklist, the quality of the included studies was judged to be high (fig 2). Most of the quality assessment items of the included studies had a low risk of bias.  5). The Spearman correlation coefficient (r=0.333, p=0.347) indicated that there was no significant threshold effect (p>0.05), which also showed that other factors may cause the heterogeneity. All the above statistical analyses results were acceptable.

Publication bias
The Deeks' funnel plot for testing publication bias showed that the studies were distributed symmetrically with a p-value of 0.10 (p>0.05), indicating no clear evidence of publication bias (fig 6).

Heterogeneity detection
The threshold effect test suggested other factors besides the threshold effect might result in heterogeneity between included studies [10,13,16,17,19,[22][23][24][25][26]. The meta-regression was adopted. The results suggested that the source of cases (China or other group), the number of masses (number ≥200 or ˂200) and the edition of BI-RADS (the fourth or the fifth) had something to do with heterogeneity (Table II). Among these parameters and categories, it was found that only the variable of the number of the masses has statistical significance for the sensitivity (p=0.04). The studies based on more than 200 masses displayed a higher sensitivity of 0.86 (95%CI: 0.83-0.90) [10,16,23]; others based on less than 200 nodules showed a little lower summary sensitivity of 0.75 (95%CI: 0.67-0.83) [13,17,19,22,[24][25][26]. As for the summary specificity, the level of the number of the masses   also has statistical significance (p=0.01). The studies based on more than 200 masses evidenced a low specificity of 0.79 (95%CI: 0.67-0.90), while others offered a higher specificity of 0.91 (95%CI: 0.83-0.98). The difference of sensitivity and specificity were obvious after the comparison of the two subgroups of data.

Sensitivity analysis
To find out if there was any research affecting the stability regarding PSEN and PSPE, we removed the studies one by one and the results of the sensitivity and specificity analysis are shown in Table III. The results showed that, excluded each single study, the PSEN and PSPE were not significantly influenced and the Higgins I 2 also did not change significantly.

Fagan plot analysis
The analysis of the Fagan plot showed that S-Detect could give some useful information about the patients' illness. The PLR and NLR values were calculated by setting the prior probabilities of 25%, 50% and 75% ( fig 7). As shown in the figure, when the pre-test probability was 50%, there was 85% probability of the correct diagnosis of breast cancer with S-Detect following the "positive" measurement and lowering the probability of breast cancer with S-Detect to 17% following the "negative" measurement (fig 7b). When the pre-test probabilities were 25% and 75%, the positive post-test probabilities were 66% and 94% while the negative post-test probabilities were 6% and 38%, respectively (fig 7a,c).

Discussions
To our best knowledge, this is the first meta-analysis that comprehensively investigates the diagnostic performance of S-Detect technique (one of the most increasingly used ultrasound CAD systems) in the differentiating of benign and malignant breast masses. All selected studies were of high quality, so it is likely that the quality of the studies did not have a significant impact on the results of the meta-analysis. In addition, it is not expected that the meta-analysis overestimates the effect of S-Detect evaluation, since there is no evidence of publication bias.   In this study, the PSEN and PSPE of S-Detect in the differentiation of breast masses were 0.82 (95%CI: 0.75-0.87) and 0.86 (95%CI: 0.73-0.93), respectively. In a study conducted by Badu-Peprah et al using US alone to diagnose breast masses [27], the sensitivity was 100% (95%CI: 93.2%-100%) higher than S-Detect, but the specificity of 80.4% (95%CI: 66.9%-90.2%) was lower compared to S-Detect. However, in comparison with the study of Kolb et al also using US alone [28], they showed completely different results where the sensitivity and specificity were 75.3% and 96.8% respectively. Apart from US, there are also some other useful imaging techniques that can be used to screen and diagnose breast cancer such as mammography and MRI which are all anatomy-and morphology-based imaging techniques [29,30]. Mammography is well known to be a powerful screening tool in the detection of early breast cancer and could reduce breast cancer related death, but it is imperfect, particularly for women with dense breasts [31,32]. MRI has become more prevalent in the last few years [33]. Previous meta-analysis showed that MRI combined with CAD system can reduce the number of misdiagno-  ses and shorten the time required to interpret the image, thus improving the accuracy of diagnosis. However, the limitations of MRI are its high cost, as well as modest specificity resulting in false positive examinations [34]. The earliest study in this meta-analysis on S-Detect technique was published in 2016 [13]. A study by Jeong et al revealed a high diagnostic sensitivity (88.6%) and accuracy (86.0%) of S-Detect, which was not statistically different from an experienced radiologist (84.1% and 91.0%, respectively) but the specificity was significantly lower than the experienced radiologist (83.9 vs. 96.4%) [35]. However, operator dependency needs to be improved in this study.
All the 10 included articles use the BI-RADS to make the final assessment, but they did not use the same cut-off value. BI-RADS 4a was clearly described as the cut-off value in 7 [10,19,[22][23][24][25][26] but not in 3 studies [13,16,17]. The 10 studies used different indications for breast masses and there was a wide variation in the number and pathological type of focal lesions selected [10,13,16,17,19,[22][23][24][25][26]. The methods for obtaining pathological results are also inconsistent. So, they may have a high heterogeneity inevitably. Therefore, a random-effects model was used to interpret the results with more caution. First, meta-regression was selected to explain the high heterogeneity. It is obvious from the results that the total number of the breast masses may be an important reason for the heterogeneity. 5 included articles had more than 200 masses to differentiate benign or malignant masses [10,16,23,25,26], while other 5 articles had less than 200 masses [13,17,19,22,24]. The p-value of the sensitivity and specificity were 0.04 and 0.01, respectively, which indicated that the result have certain statistical significance. It is demonstrated that the total number of the breast masses in studies may be a cause of heterogeneity. As can be seen from the results, articles which had more than 200 masses [10,16,23,25,26] may have a higher sensitivity than those with less than 200 masses [13,17,19,22,24] (0.86 vs. 0.79) and have a low specificity than those with less than 200 masses [13,17,19,22,24] (0.79 vs. 0.91). Perhaps later studies can further assess the impact of sample size on results by increasing the masses number included. Other possible causes of high heterogeneity have also been analysed. Among the articles included in the analysis, 7 were from China [10,13,16,[23][24][25][26], the remaining 3 were from other countries including Korea [17,22] and Egypt [19]. The results showed no significant difference in specificity and sensitivity. S-Detect has emerged as a useful tool in the field of medical US over the past few years [9]. But so far, only four countries have conducted clinical trials on this technique including China [9,10,13,16,23,24], South Korea [17,18,22], Egypt [19] and Italy [36,37].
US BI-RADS is considered as a standardized system of reporting the possibility of cancer of a breast lesion imaged on US [38]. In this analysis, 6 of the included studies used the fifth version of the reporting system [10,16,19,22,23,26] while the other 3 studies used the fourth [13,17,24]. The regression analysis showed that there was no significant difference in the results of the study (p>0.05). Still, we could not ignore the impact of version differences on the results because the fifth version of BI-RADS expanded and normalized the contents of the fourth edition, which might lead to differences in the diagnosis of masses [38,39]. And both the sensitivity and specificity of the fifth edition are higher than the fourth edition (0.84 vs. 0.78 and 0.87 vs. 0.86).
What is more, the Fagan plot analysis also has been adopted to explore the clinical practical application of the S-Detect technique. Results indicated that S-Detect had the potential to identify breast malignant masses. Considering a patient who is estimated clinically to have a 50% probability of breast cancer after the initial assessment, the likelihood that this patient has breast cancer if S-Detect is positive increases from 50 to 85%. This high probability would be considered good performance. By contrast, if the result of S-Detect was negative, then the probability that this patient has breast cancer is 17%, which is good to rule out breast cancer with confidence. In actual clinical practice, masses with a pre-test probability of malignancy of 25% would undergo biopsy, regardless of the results of S-detect. But the Fagan plot analysis suggests that S-detect could be a useful aid in providing doctors with diagnostic advice.
Five of the included studies compared the accuracy of diagnosis of breast masses among radiologists with different work experience and S-Detect，concluding that S-Detect is especially helpful for younger radiologists [10,13,16,22,24].
In this meta-analysis, we tried to collect all published studies concerning the S-Detect technique by using extensive and comprehensive search terms and multiple databases, then carried out rigorous evaluation and analysis one by one, and further statistically processed the data through quantitative synthesis to reach comprehensive conclusions. Meanwhile, a relatively objective evaluation of S-Detect technique for breast cancer can be performed by analyzing the heterogeneity of different studies.
This diagnostic meta-analysis has several limitations. S-Detect technique has also been applied in Italy [36,37], but these two articles were excluded for their own reasons. The study done by Bartolotta et al has no data for assessing the diagnostic performance of S-Detect alone but this article showed that both inter-and intra-observer agreement of two residents improved after the use of S-Detect, with k values ranging from 0.7 to 0.81 (from good to very good, without statistical significance) [36]. Moreover, the study done by Di Segni et al only selected lesions classified as BI-RADS 4a for further inspection and comparison using S-Detect technique [37]. It also proved that S-Detect is a feasible tool for the characterization of breast lesions and has a potential as a teaching tool for the less experienced operators. Second, the present analysis included a relatively small number of studies (n=10). These factors above may influence the assessment of diagnostic performance of S-Detect. Finally, all articles included were single-centre studies, the research results may have monotonicity and limitations. To further evaluate the role of S-Detect, large, prospective, international, multicentre studies in various regions are necessary.

Conclusion
Based on the above, we have obtained reliable evidence that artificial intelligence technology applied to ultrasound has developed rapidly and the S-Detect technique is becoming a useful supplement to traditional ultrasound in clinical practice, especially for inexperienced doctors. Also, this analysis can be used as a basis for further investigation on the S-Detect technique.