{'fraction_non_alphanumeric': 0.03362064044751413, 'fraction_numerical': 0.015754323876933617, 'mean_word_length': 4.816733067729084, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Group-Attention Single-Shot Detector (GA-SSD): Finding Pulmonary Nodules in Large-Scale CT Images

Early diagnosis of pulmonary nodules (PNs) can improve the survival rate of patients and yet is a challenging task for radiologists due to the image noise and artifacts in computed tomography (CT) images. In this paper, we propose a novel and effective abnormality detector implementing the attention mechanism and group convolution on 3D single-shot detector (SSD) called group-attention SSD (GA-SSD). We find that group convolution is effective in extracting rich context information between continuous slices, and attention network can learn the target features automatically. We collected a large-scale dataset that contained 4146 CT scans with annotations of varying types and sizes of PNs (even PNs smaller than 3mm were annotated). To the best of our knowledge, this dataset is the largest cohort with relatively complete annotations for PNs detection. Our experimental results show that the proposed group-attention SSD outperforms the classic SSD framework as well as the state-of-the-art 3DCNN, especially on some challenging lesion types.


Introduction
Lung cancer continues to have the highest incidence and mortality rate worldwide among all forms of cancers (Bray et al., 2018). Because of its aggressive and heterogeneous nature, diagnosis and intervention at early stage, where the cancer manifests as pulmonary nodules, are vital to survival . Although the detection of pulmonary nodules has been improved using new generations of CT scanners, certain nodules (such as groundglass nodules, GGNs) are still misdiagnosed due to the noise and artifacts in CT imaging (Manning et al., 2004;Hossain et al., 2018). The design of a reliable detection system is increasingly needed in clinical practice.
Deep learning techniques using convolution neural networks (CNN) is a promising and effective approach for assisting lung nodule management. For example, (Setio et al., 2016) proposed a system for pulmonary nodule detection based on multi-view CNN, where the network is fed with nodule candidates rather than whole CT scans. (Wang et al., 2018a) presented a 3D CNN model trained with feature pyramid networks (FPN) (Lin et al., 2016) and achieved the state-of-the-art on LUNA16 1 . However, all these algorithms neither making use of the spatial relations between slices nor introducing the attention mechanism for learning effective features. In addition, the distribution of target pulmonary nodules and the none-PNs area consists of background and other tissues, is highly unbalanced, learning to automatically weight the importance of slices and pixels is essential in the process of pulmonary nodules detection.
In this work, to address the problem of indiscriminate weighing of pixels and slices, we propose a lung nodule detection model called group-attention SSD (GA-SSD), which leverages one-stage single-shot detector (SSD) framework (Liu et al., 2016;Fu et al., 2017a; and attention module with group convolutions. Firstly, a group convolution module is added at the very beginning of the network to weight the importance of input slices, which pre-processes context information for the input of the SSD. Secondly, the attention mechanism is integrated into the traditional SSD framework to enhance the weight of nodule's pixels on a 2D image. We evaluate the proposed system on our challenging large-scale dataset containing 4146 patients with relatively complete annotations. Different from existing datasets, the cohort contains eight categories of PNs including ground glass nodules (GGNs) which are hard-todetect lesions of clinical significance yet not usually included in conventional datasets.

Related Works
Object Detection. Recent object detection models can be grouped into one of two types (Liu et al., 2018), two-stage approaches (Girshick et al., 2014;Girshick, 2015;Ren et al., 2015) and one-stage methods (Redmon et al., 2016;Liu et al., 2016). The former generates a series of candidate boxes as proposals by the algorithm, and then classifies the proposals by convolution neural network. The latter directly transforms the problem of target border location into a regression problem without generating candidate boxes. It is precisely because of the difference between the two methods, the former is superior in detection accuracy and location accuracy, and the latter is superior in algorithm speed.
Attention Modules. The inspiration of attention mechanism comes from the mechanism of human visual attention. Human vision is guided by attention which gives higher weights on objects than background. Recently, attention mechanism has been successfully applied in NLP (Vaswani et al., 2017;Cho et al., 2014;Yin et al., 2015) as well as computer vision (Fu et al., 2017b;Zheng et al., 2017;Sun et al., 2018). Most of the conventional methods which solve the generic object detection problems neglect the correlation between proposed regions. The Non-local Network (Wang et al., 2018b) and the Relation networks (Hu et al., 2018) were translational variants of the attention mechanism and utilize the interrelationships between objects. Our method is motivated by these works, aiming at medical images, to find the inter-correlation between CT slices and between lung nodule pixels.
Group Convolution. Group convolution first appeared in AlexNet (Krizhevsky et al., 2012). To solve the problem of insufficient memory, AlexNet proposed that the group convolution approach could increase the diagonal correlation between filters and reduce the training parameters. Recently, many successful applications have proved the effectiveness of group convolution module such as channel-wise convolution including the Xception (Szegedy et al., 2015) (Extreme Inception) and the ResNeXt (Xie et al., 2017).

Methodology
In this section, we present an effective 3D SSD framework for lung nodule detection because the detection of lung nodules relies on 3D information. The proposed framework has two highlights: attention module and grouped convolution. We call this new model group attention SSD (GA-SSD) because we integrate group convolution with attention modules.

Overall Framework
The proposed 3D SSD shares the basic architecture of the classic SSD (Liu et al., 2016). And the network structure of the GA-SSD can be divided into two parts: the medical image loading sub-network for reading CT scans and the backbone sub-network for feature extraction. Specifically, we use the deeply supervised ResNeXt  structure as the backbone and add GA structure to both medical image loading sub-network and backbone sub-network respectively.

SSD Architectures
The classic SSD network was established with VGG16 (Simonyan and Zisserman, 2014). Compared with Faster RCNN (Ren et al., 2015), the SSD algorithm does not generate proposals, which greatly improves the detection speed. The basic idea of SSD is to transform the image into different sizes (image pyramid), detect them separately, and finally synthesize the results.
In this work, the original stem of the convolutional layers of ResNeXt is changed into FPN-like layers. In order to detect small object, four convolution layers are added to construct the network structure ( Figure 1). The output (feature map) of different convolution layers is substituted by these FPN-like convolution layers convoluted with two different 3 * 3 convolution kernels, resulting in one output class confidence (each default box generates several class confidences) and one output regression localization (each default box generates four coordinate values (x, y, w, h)). In addition, the backbone network uses 3D convolution with same padding instead of the conventional 2D convolution layer for the group of 3D translations. And the Rectified linear unit (ReLU) is employed as activation functions to the nodes. Additionally, we apply dropout regularization (Srivastava et al., 2014) to prevent complex node connections.

GA Modules
In order to utilize the three-dimensional information between lung CT slices, we modify the basic structure of SSD network to improve the performance of the detection framework. To imitate the usual viewing habits of radiologists who usually screen for nodule lesions in 2-D axial plane at the thin-section slice, we propose a new medical imaging group convolution and attention-based network (GA module (Figure 2)) to tell the model which slices of the patient to focus on and automatically learn the weight of these slices. See figure 2, assume that the output feature map of the upper layer has N channels, which means that the upper layer has N convolution kernels. Suppose the number of group convolutions is M. So the operation of this group convolution is to divide channels into M parts. Each group corresponds to N/M channels and is convoluted independently. After each group convolution is completed, the output of each group is concatenated as the output channel. And the attention mechanism based on sequence generation can be applied to help convolutional neural networks to focus on non-local information of images to generate weighted inputs of the same sizes as the original inputs.
The GA behavior in Eq.(1) is due to the fact that all pixels (∀i) are considered in the operation. Where X represents the input signal (pictures, sequences, videos, etc., and possibly their features), Y represents the output signal with the same size as X. f (x i , x j ) is used to calculate the pairwise relationship between target i and all other associated pixel j. This relationship is as follows: the farther the pixel distance between i and j is, the smaller the f value is, indicating that the j pixel has less impact on i. g(x) is used to calculate the eigenvalues of the input signal at the j pixel. C(x) is a normalized parameter.
(1) Figure 2: The architecture of GA module, it can be divided into two parts: The former enforces a sparsity connection by partitioning the inputs (and outputs) into disjoint groups. The latter use the concatenated groups to find the non-local information.
Our approach improves parameter efficiency and adopts group convolution and attention module, where group convolution acts to find the best feature maps (i.e., highlight several slices from the input CT scans), and the attention module acts to find the location of the nodule (i.e., the size and shape of the target nodule in a specific feature map).
In addition, due to the simplicity and applicability of GA module, it can be easily integrated into a standard CNN architecture. For example, we apply this module not only to the data loading sub-network but also the feature extraction stage of the network, which allows the model to automatically and implicitly learn some correlated regions of different features, and focus on areas that the model needs to focus on.

A Large-scale Computed Tomography Dataset
A cohort of 4146 chest helical CT scans was collected from various scanners from several centers in China. Each chest CT scan contains a sequence of slices. Pulmonary nodules were labeled by experienced radiologists after evaluating their appearance and the sizes. They are divided into eight categories: calcified nodule with two different sizes, pleural nodule with two different sizes, solid nodules two different sizes, ground-glass nodule divided into pure-GGN and sub-solid nodules (mixed-GGN) as shown in (Figure 4). To the best of our knowledge, the current dataset is the largest cohort for PNs detection and with eight categories and varied sizes of annotated PNs. The detection of ground-glass nodules is important and challenging in clinical practice; however, it is not included as a part of the PNs detection task in conventional datasets such as LUNA16.
The images in our study were acquired by different CT scanners with Philips, GE, Siemens, Toshiba. All the chest CT images were acquired with the axial plane at the thin-section slice spacing (range from 0.8 to 2.5 mm). The regular dose CT scans were obtained at 100 kVp-140 kVp , tube current greater than 60mAs, 512 * 512 pixel resolution; and the low-dose CT images were obtained that tube current less than 60mAs with all other acquisition parameters the same as those used to obtain the regular dose CT. For our experiment, we randomly selected 80% of patients as training set and 20% of patients as testing set. Gradient updates were computed using batch sizes of 28 samples per GPU. All models were trained using the SGD optimizer, implemented batch normalization, and data augmentation techniques. Lung nodule detection performance was measured by CPM 2 . The evaluation is performed by measuring the detection sensitivity of the algorithm and the corresponding false positive rate per scan. This performance metric was introduced in (Setio et al., 2017).

Effectiveness of GA Module for Data Load
The Table 1 investigates the effects of different input methods. The baseline (multi-channel) used continuous slices as input and our approach added the GA module to this input mode. Using the GA module, our approach improved the CPM from 0.615 to 0.623. When the input changed the 3D multi-channel (not 3D patch, by reshaping the 2.5D data to (batch, 1, z, x, y)), the results improved by (+0.018) vs (+0.008). These results verified the effectivess of our usage of the GA module for data pre-processing. In addition, our approach can better promote the effect of the model on 3D data than on 2D data. For detecting small objects, feature pyramid is a basic component in the system. In general, small lung nodules are challenging for detectors, i.e., there are numerous small nodules which are 20 * 20 pixels in the 512 * 512 image data, making it difficult to localize. Thus, FPN is an important component in our framework for detecting small nodules. In order to better investigate the impact of FPN on the detector, we conducted relevant experiments on a fixed set of feature layers. From Figure 1, we can see that on the basis of the original SSD (C2, C3, C4, C5, C6), the feature map of the latter layer uses upsampling to enlarge the size, and then adds with the former layer (original FPN). In the GA-FPN version we use the GA module of the feature map as shown in Figure 2 to get the weight between feature maps. We choose to calculate the candidate box (like SSD) for several of the layers on the FPN. The lower layer, such as the P1 layer, has better texture information, so it is associated with good performance on small targets that can be identified by the detector. The higher level has stronger semantic information, and is associated with better results for the category classification of nodules. For the sake of simplicity, we do not share feature levels between layers unless specified. Table 2 compares the effect of FPN and GA-FPN across the feature maps. According to Table 2 (left) results, when feature is used at a very early stage (like P1), it brought more false positives and harmed the performance (drops from 0.654 to 0.473). However, compared to the output of the P4 layer alone, the model with lower information of the P3 layer gained relatively better performance.
According to Table 2 (right) results, the framework improved the overall performance across feature layers. That the framework performance improved from 0.696 (best in FPN ) to 0.721 (best in GA-FPN) validated our conjecture that using the GA module could help the model learn more important feature layers. Extensive experiments were performed on our large CT dataset. We mainly compared our approach with current state-of-the-art methods for object detection in computer vision fields such as RCNN (Ren et al., 2015;Xie et al., 2017) and SSD (Liu et al., 2016) as well as current state-of-the-art method for PNs detection. The results are mainly summarized in Tabele 3 and the other detail components can be found as follow.
From Table 3 3 , we can observe that our system has achieved the highest CPM (0.733) with the fewest false positives rate (0.89) among this systems, which verifies the superiority of the improved GA-SSD in the task of lung nodule detection. On the classes of p.ggn and m.ggn, which are challenging to detect in clinical practice, our GA-SSD outperforms other approaches by a large margin.

Conclusion
In this paper, we proposed a novel group-attention module based on 3D SSD with ResNeXt as backbone for pulmonary nodule detection with CT scans. The proposed model showed superior sensitivity and fewer false positives compared to previous frameworks. Note that the higher sensitivity obtained, the more false positive pulmonary nodules resulted. Our architecture was shown to tackle the problems of high false positive rate caused by improving recall.
In the lung cancer screening step, radiologists will generally take a long time to read and analyze CT scans to make correct clinical interpretation. But there are many factors making experienced radiologist prone to misdiagnosis, such as multi-sequence /multi-modality of images, the tiny size and low density of some lesions (such as GGN) that signal early lung cancer, heavy workload, and the repetitive nature of the job. Our proposed CNN-based system for pulmonary nodules detection achieve state-of-the-art performance with low false positives rate. Moreover, our proposed model takes only nearly 30s to detect pulmonary nodules, and it still has potential to further speed up the detection process when more computing resources are available.