{'fraction_non_alphanumeric': 0.02812898735645517, 'fraction_numerical': 0.00875768472334996, 'mean_word_length': 4.909184372858122, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Multi-Task Lung Nodule Detection in Chest Radiographs with a Dual Head Network

Lung nodules can be an alarming precursor to potential lung cancer. Missed nodule detections during chest radiograph analysis remains a common challenge among thoracic radiologists. In this work, we present a multi-task lung nodule detection algorithm for chest radiograph analysis. Unlike past approaches, our algorithm predicts a global-level label indicating nodule presence along with local-level labels predicting nodule locations using a Dual Head Network (DHN). We demonstrate the favorable nodule detection performance that our multi-task formulation yields in comparison to conventional methods. In addition, we introduce a novel Dual Head Augmentation (DHA) strategy tailored for DHN, and we demonstrate its significance in further enhancing global and local nodule predictions.


Introduction
Lung cancer ranks among the top causes of cancer-related deaths worldwide. Pulmonary nodule findings, though typically benign, are an alarming sign for potential lung cancer. Given its simple and inexpensive operating cost, chest radiography (i.e., x-rays) is the most widely adopted chest imaging solution available. However, one concern during radiograph analysis is the proportion of nodules thoracic radiologists often miss due to the nature of the imaging modality [5,26]. A chest radiograph is a 2D projection of a patients' chest. Thus, nodules appear less visible when occluded by other organs (e.g., rib cages) or foreign bodies (e.g., CVADs). With the rising workload already posing a challenge for thoracic radiologists, assistive tools to reduce missed nodules during chest radiograph analysis are gaining significant clinical relevance [4,10,19].
To identify pulmonary nodules on chest radiographs, several works propose potential solutions. Some [1,21,28] focus on image-level prediction indicating nodule presence per scan (we refer to as global methods). Others [11,13,18,24] study patch-level prediction exploring nodule detection with local bounding box information for each nodule (we refer to as local methods). Although both local and global methods offer information regarding nodule presence in a given chest radiograph, adopting just a single method alone can be undesirable. Local methods offer the benefit of pinpointing each nodule, but the adopted labeling criterion can be highly subjective and prone to inconsistency [2,12]. Global methods alleviate this issue by predicting a single label indicating nodule presence, but further effort is required by the examiner to locate these nodules.
Li et al. [14] and Pesce et al. [20] attempted to address this disparity. The former formulated a multiple instance learning (MIL) model to classify nodule presence using a grid of patches across the input scan. The image-level label is then computed using the joint probability across patch predictions. Despite its attempt in combining local and global predictions, such MIL model is not translation equivariant by design, causing the predicted global label to be highly dependent on the nodule location innate to the scan. The latter proposed CONAF, a network composed of a shared backbone between its classification and localization head. The classification head outputs a global label indicating nodule presence, and its localization head outputs a downsized score map. Considering that most nodule sizes are relatively small with respect to the scan size, the low resolution scoremap limits the localizer head from explicit localization purposes.
In this work, we present a novel multi-task lung nodule detection algorithm using a Dual Head Network (DHN) and an accompanying Dual Head Augmentation (DHA) training strategy. Our multi-task objective is similar to [3], but instead of using a RetinaNet [16], we adopt a modified Faster-RCNN [22] architecture customized for nodule detection. Considering the properties of pulmonary nodule, we propose to use deformable convolutions [6] and the gIOU loss [23]. In addition, we incorporate our novel DHA strategy during DHN training, and we demonstrate its importance in further enhancing global and local nodule predictions. The remaining of the paper is organized as follows. We first illustrate the preliminaries in Section 2. The proposed methods will be detailed in Section 3, followed by experimental results in Section 4. Conclusion will be given in Section 5.

Preliminaries
The Faster-RCNN [22] is a two-stage network originally designed for object detection. The first stage of the network is a feature extractor (i.e. VGG-16 [25]), and the second stage consists of an RPN, ROIPool, and ROIHead module. For a given input image, the feature extractor outputs a feature representation of that image, and this representation is fed to the second stage. The RPN generates bounding box proposals around regions that contain potential non-background objects. Crops for each proposed region are then taken from the extracted feature representation, and they are resized to a fixed dimension using ROIPool. The fixed-size feature crops are then independently classified using the ROIHead, and an updated bounding-box is predicted for each crop.
Modern implementations of the Faster-RCNN often include two modifications to the original design that enhances detection performance. The first is the attachment of the Feature Pyramid Network (FPN) [15]   resolution representations which is beneficial for detecting small objects. The second is the replacement of ROIPool with Multi-Scale ROIAlign [7,15] to avoid quantization while cropping the multi-resolution features. During training, the RPN generates an objectness loss obj from the background foreground classification and a regression loss reg from the distance computed between the proposed regions with their matched ground truth bounding boxes. The ROIHead generates a classification loss cls during feature crop classification and a bounding box loss bbox from the distance computed between the updated bounding boxes and their matched ground truth bounding boxes. The default setup utilizs a Smooth L1 Loss to train reg , bbox and the Cross Entropy Loss to train obj , cls . The final loss function L local is formulated in Equation 1 L local = obj + reg + cls + bbox . (1)

Methods
In this section, we present our multi-task Dual Head Network (DHN) architecture and the accompanying Dual Head Augmentation (DHA) strategy. The DHN takes advantage of the two-stage structure seen in the Faster-RCNN by adding an additional network in parallel to the original second stage. The DHA strategy is applied during training, and it utilizes the DHN's dual head design in improving nodule detection performance. The specifics are detailed in the following sections.

Dual Head Network
The DHN architecture is designed in a two-stage approach (see Figure 1). The first stage is a feature extractor with FPN, and the second stage consists of two parallel networks that we refer to as the global head and the local head. For a given scan, the feature extractor first extracts its respective representation. Then, the global head predicts a binary label indicating nodule presence, and the local head predicts bounding boxes around each detected nodule. The feature extractor of our DHN is a modified ResNet-18 [8]. We conduct a series of experiments comparing various CNN architectures from the ResNet family, and we observe slightly better performance as model size increased. However, the training time increases significantly for larger models. Hence, we select the smallest model from the ResNet family, the ResNet-18, to serve as a lower bound for DHN performance throughout our experiments. We modify the ResNet-18 by replacing standard convolutions with deformable convolutions [6] in the final three layers. Applying deformable convolutions allows more dynamic focus on particular regions in the image where the nodules size might be small (see Figure  2). For a given scan x, we take the intermediate representations the ResNet-18 extracts, and we pass them to the FPN to obtain set of multi-resolution representations. This set of multi-resolution representations is fed to the global and local heads for further processing.
The global head 's primary purpose is to classify whether nodules are present in a given scan. Thus, we select the representation with the largest receptive field [17] as input. As shown in Figure 3, this representation is passed through two consecutive 2D convolutions and ReLUs before being max-pooled into a single vector. Then, a linear layer and a softmax layer are applied to obtain the probability indicating nodule presence in the scan. For a set of N labeled scans is the i th scan with resolution h × w (we set h, w = 512) and y i ∈ {0, 1} is the corresponding global label, we formulate the global head loss L g (see Equation 2) using a weighted cross-entropy: where α 1 and α 2 are two hyperparameters. The local head serves as a nodule detector, and we adopt the second-stage design as specified in Section 2. During training, however, we replace the Smooth L1 ) is a weighted sum of the two losses, i.e.,

Dual Head Augmentation
Data augmentation is a well-known technique that increases diversity in the training data to improve model generalization. In classical image classification or object detection tasks, one augmentation strategy (i.e., a pre-defined set of stochastic transforms) is applied per training image during the forward-pass. However, training with just one augmentation strategy per image for a dual head architecture can easily lead to one head being particularly optimized while the other head performs mediocrely.
To fully optimize each head to their specific objectives, we propose a novel data augmentation strategy called the Dual Head Augmentation (DHA). DHA takes advantage of the DHN's dual head structure by applying an augmentation strategy for each head. As shown in Figure 3, we designate an augmentation function φ g for the global head and φ l for the local head. Given an input scan x, the augmented images φ g (x) and φ l (x) are generated and batched together. This batch is fed into the feature extractor with FPN, and we obtain the multiresolution representations. We split the batch, and we feed the representations corresponding to φ g (x) and φ l (x) to the global and local heads respectively to optimize each head.
We select a set of image transformations 1 that involves illumination transforms (e.g., histogram equalization, random brightness, etc.) and geometric trans- forms (e.g., horizontal flip, rotation, etc.). In this work, we consider two types of augmentation strategies that exhibit favorable single-head performance, and we define the augmentation strategies by their sampling method. The first sampling method is binomial sampling. Formally, for a given probability p and a set of transforms Θ = {θ j } M j=1 , binomial sampling selects transform θ j for j ∈ {1, ..., M } with probability p, and we refer to the corresponding augmentation strategy as φ bin (x; p, Θ). The second sampling method is uniform sampling. Specifically, uniform sampling selects a transform from Θ with probability 1/M , and we refer to this augmentation strategy as φ uni (x; Θ). During each training iteration, the augmentation strategy samples from Θ using one of the above sampling methods and applies the selected transforms.

Experiments
In this section, we evaluate the nodule detection performance of a DHN in comparison to other notable approaches. In addition, we perform a series of experiments to compare the influence different augmentation strategies play towards enhancing global and local predictions. Classification performance is evaluated using the global head prediction, and localization performance 2 is evaluated with local head predictions. We report on the test set mean and standard deviation of the top 8 performing validation checkpoints throughout each experiment. Each model is trained with SGD and momentum for 70 epochs (∼3 days on an NVIDIA P100 16GB GPU) with a step size of 5e-5 and momentum of 0.975 at a batch size of 2. Loss weights α 1 , α 2 are set to 0.69 and 1.76 while λ 1 , λ 2 are set to 0.35 and 2.5.

Dataset
The dataset we evaluate in this study is a subset of the National Institute of Health (NIH) Chest X-Ray dataset [28]. Each case in our dataset was labeled with bounding box annotations for each nodule by three thoracic radiologists. Prior to labeling, each radiologist had to pass an assessment test requiring them to correctly identify over 80% of the cases from a held-out test set with potential lung nodules. The radiologists were then asked to independently label each case in our dataset. Labels were aggregated upon completion, and the radiologists reviewed each case to reach a consensus on which labels to keep. A final senior radiologist then reviewed and control the quality of annotations. In total, we randomly sample 26000 scans from NIH dataset and 21,189 qualified scans are added to our dataset. These scans are split into training, validation, and test sets following the ratio of 80 : 10 : 10 based on patient identifiers.

Dual Head Network Analysis
For our first set of experiments, we evaluate the performance of a DHN in comparison to several global and local methods [9,16,27]. We also compare our DHN with a notable dual-head approach CONAF [20]. Although CONAF's localization head generates a heatmap-like mask for nodule localization, the mask is too coarse to derive precise bounding box coordinates from. Hence, we only evaluate the classifier head's prediction in our comparison. For fair comparison, we do not apply data augmentation to the training strategy. We also analyze the single head performance of the DHN. Specifically, we consider the case where we train only the global head and the case where we train only the local head.
As shown in Table 1, the DHN architecture yields favorable improvements in both classification and localization metrics. We believe this improvement is that our DHN extracts more informative representations due to cross-task supervision. Specifically, training with local labels using the local head encourages the feature extractor to extract more meaningful representations that highlight local findings. Since these representations are shared with the global head, classification can benefit from the available local information the representations possess. Conversely, global labels are trained with high level features that have the widest receptive field. This encourages the feature extractor to leverage potential ancillary information beneficial for global prediction which may also assist local predictions. As our experiments demonstrate, we observe complementing nodule detection performance from the two heads when they are jointly trained.

Dual Head Augmentation Evaluation
In this subsection, we are interested in analyzing the effects different augmentation strategies impose on the DHN's global and local predictions. As mentioned in Section 3.2, we consider both binomial and uniform augmentation strategies φ bin and φ uni . We also include an identity transform (no augmentation) strategy denoted φ id as a baseline reference.
We first analyze the performance of a DHN when only one augmentation strategy is applied on one of the two heads. From the results shown in Table  2, we can see that better classification performance is observed when the global head uses the strategy φ bin . Similarly, we observe slightly better localization   Table 3. Comparison of dual head DHA strategies (i.e., φ bin and φ uni applied on the global and local heads).
With the observed characteristics for the global and local head, we propose a hypothesis that heavier augmentations φ bin are more suitable for the global head but not necessarily the local head. We believe that the global head favors heavier augmentations to increase its robustness against image-level distortions. In contrary, the local head seems to prefer lighter augmentations as heavy augmentations might have imposed an overwhelming amount of distortions to region specific features. We verify our hypothesis with an experiment shown in Table  3. The results verify our hypothesis that the DHN obtains optimal performance when the DHA strategy applies φ bin on the global head and φ uni on the local head.

Conclusion
In this work, we present a multi-task lung nodule detection algorithm using a DHN. Our DHN architecture features a global head and a local head that performs lung nodule detection on the global and local level simultaneously. In addition, we introduce a novel DHA strategy that leverages the dual head design of the DHN to enhance global and local nodule detection performance while training. Throughout our experiments, we demonstrate the performance gain our DHN yields in comparison to conventional single head networks in both classification and localization abilities. Furthermore, we identified the DHA strategy that applies the appropriate augmentations for each head. Together with the optimal DHA strategy, our DHN obtained a performance otherwise not attainable if regular single head augmentation strategies are employed.