{'fraction_non_alphanumeric': 0.029695413862485257, 'fraction_numerical': 0.014362034274613196, 'mean_word_length': 5.008336807002918, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Automatic Polyp Segmentation with Multiple Kernel Dilated Convolution Network

The detection and removal of precancerous polyps through colonoscopy is the primary technique for the prevention of colorectal cancer worldwide. However, the miss rate of colorectal polyp varies significantly among the endoscopists. It is well known that a computer-aided diagnosis (CAD) system can assist endoscopists in detecting colon polyps and minimize the variation among endoscopists. In this study, we introduce a novel deep learning architecture, named MKDCNet, for automatic polyp segmentation robust to significant changes in polyp data distribution. MKDCNet is simply an encoder-decoder neural network that uses the pre-trained ResNet50 as the encoder and novel multiple kernel dilated convolution (MKDC) block that expands the field of view to learn more robust and heterogeneous representation. Extensive experiments on four publicly available polyp datasets and cell nuclei dataset show that the proposed MKDCNet outperforms the state-of-the-art methods when trained and tested on the same dataset as well when tested on unseen polyp datasets from different distributions. With rich results, we demonstrated the robustness of the proposed architecture. From an efficiency perspective, our algorithm can process at (approx 45) frames per second on RTX 3090 GPU. MKDCNet can be a strong benchmark for building real-time systems for clinical colonoscopies. The code of the proposed MKDCNet is available at https://github.com/nikhilroxtomar/MKDCNet.


I. INTRODUCTION
Colorectal cancer (CRC) is the second leading cause of cancer-related death and the third leading common cause of cancer worldwide [1]. The five-year survival rate is 90% for 39% of the patients that are diagnosed with localized stage disease but declines to 71% and 14% once diagnosed with regional and distant stage respectively [2]. Colonoscopy is considered the primary technique for colon cancer screening because it offers both detecting and removal of the polyp in a single operation. U.S. Preventive Services Task Force recommends forty-five to be considered as the new fifty for screening of CRC [3]. Colonoscopy can reduce the mortality through early detection at treatable stage and remove precancerous adenomas [4], [5].
During the colonoscopy operation, the average miss rate of the polyp is around 22-28% [6]. It is mainly because colonoscopy is an operator-dependent procedure and high inter-observer variations are seen in endoscopists' skills in detecting polyps [7]. During routine colonoscopy, the most frequently missed polyps are flat and smaller polyps [8]- [10]. Studies have shown that even a 1% increase in adenomas detection leads to a 3% decrease in the risk of interval colon cancer [11]. Therefore, it is highly critical to decrease the polyp miss-rate via an automated systems for CRC screening.
A Computer-Aided diagnosis (CADx) can highlight the suspicious frames and improve colonoscopy procedures. Jha et al. [12] proposed DoubleU-Net that used two U-Net's where the output of first U-Net acts as a soft-attention to the other. The network uses VGG-19 as an encoder and efficient blocks such as squeeze and excitation network [13] and atrous spatial pyradimal pooling [14] to capture some semantically meaningful information. DoubleU-Net showed state-of-the-art (SOTA) results on different biomedical image segmentation datasets. Wu et al. [15] proposed a lightweight context-aware network, PolypSeg+, for real-time polyp segmentation. The proposed architecture can capture distinguishable polyp features even with less trainable parameters and retain real-time speed. Tomar et al. [16] proposed a feedback attention network (FANet) for improved biomedical image segmentation, where they showed the SOTA performance on seven publicly available benchmark datasets. FANet unifies the mask of the previous epoch with the current training epoch and rectifies the prediction iteratively during the test time for improved performance. Ji et al. [17] proposed a progressively normalized self-attention network (PNS-Net) for video polyp segmentation. Shen et al. [18] proposed a hard region enhancement network (HRENet) for automatic polyp segmentation.
Despite the several automated methods proposed to improve the accuracy of polyp segmentation, further investigations are required to show the generalizability of the existing and the proposed method. Currently, most of the algorithms are only trained and tested on the same datasets [16], [17], [19]- [21]. Therefore, we aim to develop a novel deep learning algorithm to work well on varying distribution datasets coming from different institutions across different countries. To this end, we introduce a multiple kernel dilated convolution network (MKDCNet) architecture and test its performance on four still image datasets (polyps) and one cell nuclei dataset.
The main contribution of our work can be summarized as follows:  output for more robust feature representation that helps in accurate polyp segmentation. 2) We obtained SOTA results on four publicly available polyp datasets (same train-test set), and a nuclei segmentation dataset. Similarly, the proposed method outperformed other methods on three cross-center polyp dataset. Extensive experimental results shows the strong learning and generalization ability of MKDCNet.

II. METHOD
The proposed MKDCNet architecture is illustrated in Figure 1. The architecture begins with a pre-trained ResNet50 [22] as the encoder from which we extract four different feature maps. Each of these feature map is then passed through a sequence of 3 × 3 convolution layer, batch normalization, and a ReLU activation function. The output from the ReLU activation function is then passed through our novel Multiple Kernel Dilated Convolution (MKDC) block, which consists of multiple parallel convolution layers with different kernel sizes and dilation rates. After that, we have three decoder blocks, the output from all the three decoder blocks is passed through a Multiscale Feature Fusion (MSFF) block where we upsample and fuse the feature map to produce a more robust semantic representation. Finally, this feature map is then passed through a 1 × 1 convolution followed by a sigmoid activation function generating a binary segmentation mask.

A. Multiple kernel dilated convolution (MKDC) block
The MKDC block begins with four parallel convolution layers with a kernel size of 1 × 1, 3 × 3, 7 × 7 and 11 × 11 respectively. The kernel size's progressive increase helps capture a broad range of features, allowing the network to learn a more robust representation. Each convolution layer is then followed by batch normalization and a ReLU activation function. Next, each of these feature maps are then concatenated and passed through four parallel convolution layer, each having a dilation rate of 1, 3, 7 and 11, respectively. The use of different dilated convolutions helps to further expand the field of view and allows the network to capture more details and refine the significant features. In this sense, the MKDC is similar to multi-resolution strategies but in our case we capture rich details with convolutional kernels instead of using multiple parallel architectures or iterative and simultaneous connection from each resolutions. Each of the convolution layer is then followed by batch normalization and ReLU activation function. After that, we perform a concatenation over these features and feed them to a 1 × 1 convolution followed by a residual connection. Finally, the generated feature maps are passed through a channel and spatial attention mechanism which further highlight the significant features.

B. Decoder block
The decoder block begins with a bilinear upsampling which increases the spatial dimensions (height and width) of the input  [26] 200 Variable Colonoscopy 2018 Data Science Bowl [27] 670 256 × 256 Nuclie feature map by a factor of two. After that, the upsampled feature map is then concatenated with the output of another MKDC block, that brings more semantic information to the decoder increasing its feature representation. Next, we have two residual block, where each residual block consists of a convolutional block and an identity mapping connecting the input and output of the convolutional block. The convolutional block begins with two 3 × 3 convolution layer, where each is followed by a batch normalization and a ReLU activation function.

C. Multiscale feature fusion (MSFF) block
We use the proposed MSFF block to enhance the feature at different scales by aggregating them to produce a more robust feature representation. The MSFF block takes the output from the first decoder block and passes it through a bilinear upsampling layer to increase its spatial dimensions by a factor of two. After that, it is followed by a 3 × 3 convolution layer, batch normalization and a ReLU activation function. The output of the ReLU activation function is then concatenated with the output from the second decoder block. Next, we again follow a bilinear upsampling layer where the concatenated feature map is upsampled by a factor of two and then followed by a 3 × 3 convolution layer, batch normalization and a ReLU activation function. The output from the ReLU activation function is then concatenated with the output from the third decoder block. After this, the feature map is again upsampled and passed through a 3 × 3 convolution layer, batch normalization and a ReLU activation function. The feature map is then passed through channel and spatial attention mechanism that focus on significant features and thus improve the feature representation and its robustness.

III. EXPERIMENTAL SETUP
In this section, we will present the datasets, evaluation metrics, and implementation details used in this study.

A. Datasets and evaluation
For this study, we have select four publicly available polyp datasets and a nuclie segmentation dataset. The details about the number of images, their size, and their application can be found in Table I. We have utilized Kvasir-SEG [23], BKAI-IGH [24], CVC-ClinicDB [25], and MedAI challenge test set [26] datasets for the polyp segmentation task. For the cell nuclei segmentation task, we have used the 2018 Data Science Bowl [27] dataset. To evaluate the performance of all the models, we have used metrics such as Dice Coefficient (DSC), mean Intersection over Union (mIoU), precision, recall, accuracy, F2-score, and Frame Per Second (FPS).

B. Implementation details
We have implemented the proposed MKDCNet and the SOTA methods using the PyTorch framework. For a fair comparison, we have used the same set of hyperparameters for all models used in this study. All models were trained on NVIDIA RTX 3090 GPU, where both the images and masks were first resized to 256 × 256 pixels for better utilization of GPU. The datasets were then split into training, validation and testing in the ratio of 80:10:10, except for Kvasir-SEG, where a split of 880/120 was used for training and testing respectively. An online data augmentation strategy was used on the training dataset which includes random rotation, horizontal flipping, vertical flipping and coarse dropout. The data augmentation helped to increase the robustness of the model. All the models were trained with an Adam optimizer having a learning rate of 1e −4 with a batch size of 16. A combination of binary crossentropy loss and dice loss was used. ReduceLROnPlateau was used while training to reduce the learning rate for better performance. An early stopping criterion was also used to stop the training when the model stops improving.

IV. RESULT
At first, we performed validation of the algorithms on same datasets (same distribution). Next, we tested the trained model on completely unseen polyp datasets from different medical centers (different distribution). Table II shows the result of the MKDCNet and SOTA methods. On the Kvasir-SEG dataset, MKDCNet achieved a DSC of 0.8887 and mIoU of 0.8267 and outperformed the most competitive benchmarking method DeepLabv3+ with ResNet50 encoder with a margin of 0.5% in DSC and 0.94% in mIoU. Similarly, MKDCNet had a higher recall, precision, F2-score and nearly equal accuracy. Both DeepLabv3+ and MKDCNet had a real-time speed. Similarly, with the BKAI-IGH [24], our method outperformed DeepLabv3+ with a margin of 0.41% in DSC and 0.78% in mIoU. Additionally, we performed experiments on the 2018 Data Science Bowl [27] dataset, where we showed that our method consistently outperforms all other baseline methods. Figure 2 showed the example of qualitative results along with the heatmap. The qualitative results indicated that MKDCNet had better segmentation as compared to the UNet [28] and DeepLabv3+ [31].   The heatmap shows the region of interest and its statistical significance and the color intensity shows the effect. The red and yellow colors denote the most significant feature and the blue color denote the least significance feature.

C. Ablation study
In Table IV, we presented the ablation study on Kvasir-SEG dataset. When we compared setting #3 and setting #4, there was a 1.02% improvement in DSC and a 1.94% in mIoU with the multiple Kernel dilated convolution and multiscale feature fusion block in the network. Similarly, the Table IV also showed an improvement over both of the individual blocks.

V. CONCLUSION
We presented a novel architecture, MKDCNet, that utilizes ResNet50 as an encoder and the novel multiple kernel dilated convolution block to learn more robust representation to automatically segment polyps from colonoscopy images with high performance. Extensive experimental results on four publicly available datasets (both on the same set as well as on completely unseen datasets) consistently showed that MKDCNet has the promising capability to improve the segmentation accuracy. With MKDCNet, we obtained a realtime processing speed of nearly 45 frames per second. Our results exhibited that MKDCNet has a better generalizability, accuracy, and real-time speed. Thus, MKDCNet can be a strong new baseline for developing artificial intelligence-based support to improve the traditional colonoscopy procedure. In the future work, we plan to exploit MKDCNet under federated learning settings where we can train multiple institute datasets and minimize the privacy concerns raised by each center.