{'fraction_non_alphanumeric': 0.031132975989426537, 'fraction_numerical': 0.015125927013730818, 'mean_word_length': 4.921739130434783, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Atrous Residual Interconnected Encoder to Attention Decoder Framework for Vertebrae Segmentation via 3D Volumetric CT Images

Automatic medical image segmentation based on Computed Tomography (CT) has been widely applied for computer-aided surgery as a prerequisite. With the development of deep learning technologies, deep convolutional neural networks (DCNNs) have shown robust performance in automated semantic segmentation of medical images. However, semantic segmentation algorithms based on DCNNs still meet the challenges of feature loss between encoder and decoder, multi-scale object, restricted field of view of filters, and lack of medical image data. This paper proposes a novel algorithm for automated vertebrae segmentation via 3D volumetric spine CT images. The proposed model is based on the structure of encoder to decoder, using layer normalization to optimize mini-batch training performance. To address the concern of the information loss between encoder and decoder, we designed an Atrous Residual Path to pass more features from encoder to decoder instead of an easy shortcut connection. The proposed model also applied the attention module in the decoder part to extract features from variant scales. The proposed model is evaluated on a publicly available dataset by a variety of metrics. The experimental results show that our model achieves competitive performance compared with other state-of-the-art medical semantic segmentation methods.


I. INTRODUCTION
Thoracic and lumbar vertebrae segmentation based on computed tomography (CT) is an essential step for many automated spine analyses, including computer-assisted surgery [1], detection of vertebral body fractures and spine deformities assessment [2]. Spine represents not only the important central axis of musculoskeletal system, but also the flexible protective shell surrounding the spinal cord, the most important nerve pathway of human body. Computed Tomography (CT) image of spine is an essential tool for the evaluation of spinal lesions, which requires advanced computerized methods to support the diagnosis, treatment and intervention of physicians. Automated spine analysis requires not only the CT scans of spine, but also neck, chest, or abdomen that accidentally cover part of the spine. Therefore, the semantic segmentation algorithm of vertebrae needs to maintain robustness for a variety of image resolutions and different spinal coverage.
A considerable amount of literature has been published on vertebrae segmentation. Recent studies for vertebrae segmentation trend to adopt deep learning technique, especially Deep Convolutional Neural Networks (DCNNs). UNet is developed by Ronneberger et al. in 2015 [3], which is one of the most prominent semantic segmentation algorithms in biomedical area. 3D-UNet is proposed as an extension of UNet for 3D volumetric segmentation by replacing all 2D operations with their 3D counterparts [4]. While some research has been carried out on vertebrae segmentation based on 2D CT slices [5], there have been few empirical investigations into the vertebrae segmentation via 3D volumetric CT scans. The encoder-decoder based semantic segmentation algorithms are still having challenges in several ways. The information of feature maps decrease during the propagation from encoder to decoder. The reception field of filter is small, which cannot handle the situation of multiscale object, as well as various image resolution. Lastly, the 3D volumetric data operation is memory-consuming, and the batch normalization has poor performance with mini-batch data.
Based on the above considerations, we propose a novel 3D encoder-decoder framework, called Atrous-ResUNet. Firstly, Atrous Residual Path is designed to alleviate the spatial loss during the propagation from the encoder to the decoder. The design of Atrous Convolution can increase the reception field of filters. Secondly, to enhance the performance of the decoder, a 3D attention module based on the dimension of volume and channel is designed. Moreover, instead of batch normalization [6], we adopt layer normalization [7] for faster and better convergence with mini-batch size under limited VRAM. Subsequently, the evaluation was applied on the spine segmentation challenge of the CSI 2014 [8]. Compared with other state-of-the-art deep learning-based methods, the proposed model shows competitive performance with a Dice coefficient of 0.8928.
The remaining part of this paper has been organized in the following way. Section II is concerned with the details of the proposed model. In Section III, we discuss the experimental setup and results. We conclude our study in Section IV.

II. METHODS
The overall architecture of proposed Atrous-ResUNet is illustrated in Fig. 1. The main contributions of the proposed model comprise three components: Atrous Residual Path, 3D Attention Module in the decoder, and Layer Normalization. Firstly, we design Atrous residual Path based on Atrous Convolution [9], ResNet [10] and Inception [11]. In the encoder, we designed a residual connection to alleviate the degradation problem. The residual encoder extracts the feature from the original CT images and the decoder with arXiv:2104.03715v1 [eess.IV] 8 Apr 2021 3D attention module reconstructs the mask for corresponding CT images.

A. Atrous Residual Path
Inspired by ResNet [10], Inception [11] and Deeplab [9], Atrous Residual Path is designed to reduce the spatial loss during the propagation from the encoder to the decoder, and alleviate the disparity between encoder and decoder. The architecture of Atrous Residual Path is shown in Fig. 1, which comprises one 1x1x1 convolution filter with dilation rate = 1, two 3x3x3 convolution filters with dilation rate = 3 and 9, respectively. For each location i on the output y and a filter w, the general expression of Atrous Convolution is defined as follows: where k denotes the kernel size, r represents the dilation rate and x is the input feature map. Besides, residual connect is also applied. Equation 2 define the general expression of residual connection.
where y is the output, and x denotes the input. F (x, W i ) is the function concatenated by 1x1x1 convolution filter and two atrous convolution filters. The operation F (x, W i ) + x is performed by a shortcut connection and element-wise addition. The output of the Atrous Residual Path from the encoder is concatenated with the corresponding up-sampling feature maps in the decoder.

B. 3D Attention Module
Based on the spatial attention and channel attention [12], we applied 3D channel attention and 3D volume attention module in the decoder to enhance the performance of decoder. As shown in Fig 1, the proposed 3D attention module is integrate with two convolution layers in the decoder to enhance the essential information on feature map with Averagepooling layers, Max-pooling layers and Sigmoid functions. The attention module comprises two different attention is applied according to the dimension of channel and volume feature maps. The input of attention model is the feature map from previous convolution layer F ∈ R V ×H×W ×C . F c avg , F c max ∈ R 1×1×1×C are the feature maps generated by 3D average-pooling layer and 3D max-pooling layer on the dimension of channel.
are the feature maps generated by 3D average-pooling layer and 3D max-pooling layer on the dimension of volume. In the 3D channel attention, the shared Multi-Layer Perceptron (MLP) is applied to the features from pooling operations to generate the 3D channel attention map M c ∈ R 1×1×1×C . W 0 and W 1 are the weights of the shared MLP, where W 0 ∈ R C/r×C , W 1 ∈ R C×C/r and r is the reduction ration, which is set as 8.
where M v ∈ R V ×H×W ×1 , F represents the generated feature map from the previous 3D channel attention, f 7×7×7 is the 3D convolution operation with a kernel size of 7×7×7, σ denotes the Sigmoid activation function, and V, H, W, C denotes volume, height, width and channel, respectively. The overall expression of the 3D attention module is summarized as follows: where F, F , F denote the input, the feature map from the 3D channel attention and the final output, respectively, and represents element-wise multiplication.

C. Layer Normalization
Layer Normalization (LN) [7] is applied in the proposed model to alleviate the issue of Batch Normalization (BN) [6] with mini-batch. BN calculates the normalized statistics according to the number of samples. In this study, the batch size is 2, and the mean and variance of the four samples cannot reflect the global statistical distribution information. Due to limited hardware resources, BN is not applicable for the batch size larger than 2. If the batch size is too small, the calculated mean and variance are not enough to represent the entire data distribution. LN is an algorithm independent of batch size. LN no longer calculates the mean and variance of all the features in mini batch, but normalizes the three dimensions of VHW along the channel dimension, which overcomes the disadvantage that BN operation is sensitive to batch size. Therefore, to address the problem of BN under mini-batch, LN is applied. Layer Normalization is defined as follows: where represents element-wise multiplication, is for numerical stability in case the denominator becomes zero by chance, γ and β are learn-able parameters.

A. Dataset
The spine and vertebrae dataset is acquired with Philips and Siemens multi-detector CT scanners by University of California [8]. The dataset includes the thoracic and lumbar spine CT scans of 10 young adults. Each scan has up to 600 slices with 512×512 resolution and 1mm inter-slice thickness. In this study, the original CT images are normalized and resized into the resolution of 256×256 with the volume of 16 slices. Among 10 patients CT scans, 9 are used for training and the last patient's CT scan is used for validation and testing. In each scan, the thoracic and lumbar vertebrae are labeled as 1, and the rest are assigned with a value of 0. To enlarge the amount of training data and make them cover broader range, the training data are overlapped for every 5 slices. Moreover, various data augmentation mechanisms are also applied to augment the training data for both original CT slices and ground truth masks. The augmentation mechanisms include erosion, dilation, horizontal flip, vertical flip, random rotate 90 degrees, elastic transform, grid distortion and optical distortion, and each method is applied to 5% training data, in a total of 40% training data.

B. Experiment Setup
The experiments are implemented under the environment of TensorFlow and Keras, and the hardware setup is Nvidia Tesla-V100 GPU with 16G Memory. For the training process of models, the batch size is defined as 2 due to the restriction of GPU memory. The learning rate and epoch are set to 0.00001 and 100 respectively for all the training processes, while Adam is the optimizer and ReLU is the activation function. The best model results are selected based on the value of validation loss. In the task of vertebrae segmentation, the ground truth only occupies a small area of the scan. In the predicted mask, the positive and negative pixels are strongly imbalanced. Therefore, instead of cross-entropy, we adopt Dice loss [13] for optimization.

C. Results and Discussion
In this study, the dice coefficient is deployed as the major evaluation method. Besides, the standard performance matrix including accuracy, precision, recall, specificity is also used for evaluation and comparison [14]. Dice coefficient is a set similarity measure function, which is usually used to calculate the similarity of two samples, and the value range is [0,1]. The segmentation performance evaluation method is defined as follows: where M gt represents the ground truth mask of vertebrae segmentation, and M p denotes the predicted mask of vertebrae segmentation. The examples of the original CT slice, the ground truth segmentation mask of vertebrae and the predicted segmentation mask are shown in Fig 2, respectively. Besides, an ablation study is conducted to compare the performance of Batch Normalization and Layer Normalization in our model shown in Table II.
To comprehensively evaluate the proposed model's performance on the task of vertebrae semantic segmentation, we compare the results with various state of the art medical semantic segmentation algorithms, including pre-trained U-Net, LinkNet, FPN with ResNet34, VGG16, MobileNet, In-ceptionv3 and DenseNet121 as backbone based on ImageNet, 3D U-Net, Dense-UNet, Residual-U-Net, 3D Dense-U-Net,  3D Residual-U-Net and MultiResUnet [15]. As shown in Fig. I, our proposed model outperforms other state-of-the-art semantic segmentation methods in terms of Dice coefficient. As shown in Table I, the proposed method achieves the best 0.8928, 0.9910, 0.9910, 0.9596 and 0.9923 in Dice, Accuracy, Precision, Recall and Specificity, respectively, outperforming other methods by a notable margin.

IV. CONCLUSION
This paper proposes a novel framework to automated segment vertebrae via CT scans, which shows competitive performance compared with other state-of-the-art semantic segmentation models. Atrous Residual Path is designed to decrease the spatial information loss during the propagation process from the encoder to the decoder, while the Atrous convolution allows the model to have a larger reception field. Besides, Layer Normalization is applied for mini-batch 3D volumetric data. In the future, we will expand the research scope to Semi-Supervised learning with Atrous-ResUNet.