{'fraction_non_alphanumeric': 0.027395722225341042, 'fraction_numerical': 0.009880424409139393, 'mean_word_length': 5.168282548476454, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Medical image super-resolution method based on dense blended attention network

In order to address the issue that medical image would suffer from severe blurring caused by the lack of high-frequency details in the process of image super-resolution reconstruction, a novel medical image super-resolution method based on dense neural network and blended attention mechanism is proposed. The proposed method adds blended attention blocks to dense neural network(DenseNet), so that the neural network can concentrate more attention to the regions and channels with sufficient high-frequency details. Batch normalization layers are removed to avoid loss of high-frequency texture details. Final obtained high resolution medical image are obtained using deconvolutional layers at the very end of the network as up-sampling operators. Experimental results show that the proposed method has an improvement of 0.05db to 11.25dB and 0.6% to 14.04% on the peak signal-to-noise ratio(PSNR) metric and structural similarity index(SSIM) metric, respectively, compared with the mainstream image super-resolution methods. This work provides a new idea for theoretical studies of medical image super-resolution reconstruction.


results.
Attention mechanism refers to that neural networks are capable of focusing on specific channels or specific regions.
According to the differences of concerns, attention mechanism can be divided into spatial attention mechanism and channel attention mechanism [20]. The cascaded channel-space attention mechanism proposed by Chen et al. [21] cascades the channel attention block and the spatial attention block together, and learns corresponding descriptors for different channels and different regions in stages, assigning different channels and different regions different weights, forcing the neural network concentrate more attention on the channels and regions with sufficient high-frequency details. Another channel attention structure proposed by Hu et al. [22] can adaptively assign different weights to different channels, enhance the channels with abundant high-frequency details, and suppress the channels with plentiful redundant low-frequency texture contents. The structure is capable of accelerating network convergence and further improves network performance. Zhao et al. indicate that the L2 norm-based loss function is a differentiable convex function [23] and obtained images by models using L2 norm-based loss function are of favorable quality and can gain high peak signal-to-noise ratio (PSNR) metric results.
In order to address the issue that medical image would suffer from severe blurring caused by the lack of high-frequency details in the process of image super-resolution reconstruction, a novel medical image super-resolution method based on dense neural network and blended attention mechanism is proposed. Our main contributions can be summarized as follows: 1. A new blended attention mechanism block is proposed. The proposed attention block learns the corresponding descriptors for input feature maps, multiplies the learned descriptors using Hadamard product with the original input, and simultaneously assigns different weights to different channels and different regions to enhance the channels and regions with sufficient high-frequency details and suppress the channels and regions with abundant low-frequency texture contents. The proposed attention block is capable of enhancing feature representation capabilities, allowing the network to simultaneously focus on channels and regions that are with sufficient high-frequency details. Compared with phased learning, the proposed block has fewer network parameters and gains higher efficiency.
2. An image super-resolution method based on dense blended attention network is proposed. The basic network structure is based on dense neural network, dense skip connections are added between the basic unit of the network and inside each basic unit, with which can fully multiplex features from different stages and different scales. Original batch normalization layers in DenseNet are removed to avoid loss of high-frequency texture details. At the end of each basic unit, the proposed blended attention mechanism block is added, so that the neural network can concentrate attention more on the channels and regions with sufficient high-frequency details, which can accelerate network convergence and further improve the performance of the network.
3. The proposed method was applied to the super-resolution reconstruction of blurry prostate cancer MRI images. Experimental results show that the high-resolution MRI images obtained by performing super-resolution with scaling ratio 2, 3, and 4 can still preserve favorable image sharpness and texture details. In addition to applying to MRI images, the proposed method can also be applied to X-ray computed tomography(CT) images, X-ray(X-ray) images, and positron emission computed tomography(PET) images super-resolution through transferring network structures, indicating favorable universality of the proposed method.

Dense neural network
The basic structure of a dense neural network is shown in Figure 1 [18]. The dense neural network is characterized by adding densely connected skip connections, fully multiplex features from different stages and different scales, and achieving better performance with less parameters and lower calculation costs than traditional residual network (ResNet). Besides, the dense skip connections allow the gradient information to be directly transmitted to any layer of the network during back propagation, greatly alleviating the problems of gradient vanishing and network degradation.
The essential difference between DenseNet and ResNet is that the input of any layer in DenseNet is derived from the output of all previous layers. Assuming the DenseNet has L layers, the output of l th layer of DenseNet x l is: where concat is the concatenation operation on the channel dimension. Equation 1 indicates that, the input of l th layer is derived not only from the output of (l − 1) th layer, but also from the output of all previous layers. Concatenate x 0 , x 1 ,x 2 ···x l−1 in channel dimension, and take the concatenated feature maps as the input of l th nonlinear transforming operator H l . However, the output of l th layer of ResNet is: Namely, the output of l th layer is only the pixel-wise sum of the output of (l − 1) th layer and the nonlinear transformation of the output of (l − 1) th layer.
In view of the fact that DenseNet is capable of fully multiplexing features from different stages and different scales, and achieving better performance with less parameters and lower calculation costs than traditional residual network (ResNet).
The proposed method in this paper is based on dense neural network.

Blend attention mechanism
Adding a blended attention mechanism block in the network forces the neural network concentrates more attention on the channels and regions with sufficient high-frequency details, which is capable of accelerating network convergence and further improves network performance. The proposed attention block cascades two convolutional layers and two activation layers, simultaneously learns corresponding descriptors for different channels and different regions, and assigns different channels and different regions different weights. Compared with phased learning, the proposed block has fewer network parameters and gains higher efficiency.

Fig.2 Blended attention block
The proposed blended attention mechanism block is shown in Figure 2. The dimensions of the input and output feature maps are both H * W * C, conv represents convolutional operations, RELU and Sigmoid are two different activation functions [24], symbol⊗ is Hadamard product [25]. Taking feature maps with dimensions H * W * C as input, after two cascaded convolutions and activations as the figure 2 shows, corresponding descriptors τ will be obtained: where W 1 , W 2 represent the parameters of the first and the second convolutional layer, respectively. δ represents RELU activation function while f represents Sigmoid activation function. The first convolutional layer performs channel downscaling with reduction ratio 16, after that, feature maps with dimensions H * W * C/16 can be obtained. The obtained feature maps are then increased by ratio 16. After dimensions down-scaling and up-scaling by two cascaded convolutional layers and two activation layers, C corresponding descriptive matrices namely descriptors τ for different channelswhere i = 0, 1, 2...C are learned. Sparser descriptive matrices are adaptively assigned to channels that contain more low-frequency texture contents, this enables the neural network concentrate more attention on the channels and regions with sufficient high-frequency details. The dimension of each descriptive matrix τ i is H * W , corresponding to each element in i th channel of the original input feature maps. After two convolutions and two activations, the channels with abundant highfrequency details are enhanced, and the channels with plentiful redundant low-frequency texture contents are suppressed.
Multiply the learned descriptor τ i using hadamard product with the i th channel to force the network focus on the regions with sufficient high-frequency details in the i th channel. In summary, feature representation through blended attention mechanism block can be obtained by multiplying the learned descriptor τ and the original input.
In view of the fact that blended attention mechanism is capable of accelerating network convergence and further improving network performance. The proposed method in this paper introduces blended attention mechanism on dense neural network.

Network structure
The basic unit of the proposed dense blended attention network in this paper is shown in Figure 3a of each convolution kernel is set to 16×3×3, namely 16 kernels whose size is 3×3. RELU is used as activation function, convolutional step size is set to 1, zero padding is used to keep the size of feature maps same [26].

Loss function
Inspired by the literature [23], we use the L2 norm-based loss to quantify the similarity between high-resolution images and high-resolution images obtained by super-resolution to guide network learning. The expression of the used loss function is: where I HR represents real high resolution images, I SR represents the high-resolution images obtained by super-resolution, H, W, C are the size and channel number of the input image, n represents the number of mini-batch learning, I v,i,j,k is the pixel value of position (i, j) in the k th channel of the v th input image. The goal of the network in the training phase is to minimize the loss function L 2 . The smaller the L 2 loss, the smaller the difference between the high-resolution image obtained by deploying super-resolution and the real high-resolution image, the better the super-resolution performance and the higher the precision.

Training details
Training HR images are cropped to sub-images of size 96×96 as preprocessing procedure. LR images with scaling ratio 2, 3 and 4 are obtained by down-sampling HR images using the MATLAB bicubic kernel function. Data augmentation is performed on the training images, which are randomly rotated by 90 • ,180 • ,270 • and flipped horizontally to obtain more training data. The mini-batch size is set to 16 due to hardware resource limitations. For optimization, the proposed method is optimized by ADAM optimizer with β 1 =0.9, β 2 =0.999, and =10 −8 [27]. The initial learning rate is set to 10 −4 and then decreases to half when there is no precision improvement in 10 consecutive epochs to achieve optimal results.

Experimental parameters settings
Training images used in this paper were prostate MRI public datasets of the University of Medicine, Nijmegen, Netherlands, which contains T2W, PD-W, DCE and DW images, all of which were derived from two different Siemens 3T MR scanners, the MAGNETOM Trio and Skyra [28]. T2-weighted images were acquired using a turbo spin echo sequence and had a resolution of around 0.5 mm in plane and a slice thickness of 3.6 mm. 400 T2W high-resolution prostate MRI images with sufficient details whose size is 384×384 were selected as training set, and 100 T2W prostate MRI images were selected as testing set. Two traditional super-resolution methods and four representative deep learning-based methods were selected as comparative experiments. The traditional methods selected were bilinear interpolation (Bilinear) and bicubic interpolation (Bicubic). The selected deep learning methods were SRCNN [12], VDSR [14], SRResNet [7] and SRDenseNet [18]. For fair comparison, each method was tested under the same hardware environment. The hardware environmental parameters of the experiment are shown in Table 1. super-resolution of single-frame image is also used as one of the quantitative evaluation metrics. For evaluation, inspired by the literature [31], this paper converted images from RGB color space to YCbCr color space [32]. All the metric results reported in this paper were computed on the y-channel after removing a 4-pixel border. The formula for calculating PSNR is: where W, H represents the image size, I HR represents real high-resolution images, I SR represents the high-resolution images obtained by super-resolution, I i,j represents the pixel value of position (i, j). The better the PSNR results, the better the image quality. The formula for calculating SSIM is: where u, σ represents the pixel mean and variance of two images for comparison, C 1 , C 2 , C 3 represents the constants to prevent the denominator from being zero. The range of SSIM is [0, 1], the closer the value is to 1, the more similar the two images. The PSNR, SSIM, and time-consuming metric results obtained by super-resolution of the testing set were averaged.
The results are shown in Table 2. Taking the quantitative evaluation results of scaling ratio 2 as an example, the proposed method has 11.247dB and 14.036% improvement in peak signal-to-noise ratio (PSNR) and structural similarity (SSIM), It can be seen from the quantitative evaluation results of scaling ratio of 2, 3, 4 that performance of the proposed method is superior to the comparative methods. Combining the performance of each method of different scaling ratio, the conclusion that the of the proposed method is effective and superior can be drawn.

Qualitative evaluation
Four sets of images with sufficient texture details were selected from the testing set to show the performance of each super-resolution method. The four images selected were MRI images of the transverse transposition of the prostate datasets. The comparisons between the high-resolution images obtained by each method and the real high-resolution images (Ground Truth, GT) are shown in Figure 4, and the corresponding quantitative evaluation results are marked below.
In overall, the images obtained by deep learning-based methods are clearer than that obtained by interpolationbased methods. SRCNN, VDSR, SRResNet, SRDenseNet and the proposed method in this paper achieved considerable performance, however, it can be noticed that more realistic texture details were produced by the proposed method in specific areas. Taking the ProstateX-0061 image shown in Figure 4 as an example, the result images obtained by bilinear and bicubic interpolation look blurry. The images obtained by SRCNN have certain improvements compared with bicubic interpolation, however, they are still blurry and over-smoothed that appear implausible. VDSR, SRResNet, and SRDenseNet achieved favorable sharpness, but blurry artifacts appear. Images obtained by the proposed method have gained better sharpness, uniform brightness, sufficient details and perception results, and are closest to the real high-resolution images.

Comprehensive evaluation
Combined with the quantitative and qualitative evaluation results, the method proposed in this paper has the characteristics of high precision. The quantitative results calculated by performing the proposed method on the testing set are higher than that of traditional interpolation-based method and the deep learning-based SRCNN, VDSR, SRResNet, and SRDenseNet methods. It can be noticed from the comparison of qualitative results of each super resolution method in Figure 4 that MRI images obtained by the proposed method have gained higher definition, better sharpness, uniform brightness, sufficient details and favorable perception results and are closet to the real high-resolution images. When it comes to the time-consuming, the proposed method improves performance with sacrifice of time, however, the consumed time is also within the acceptable range. In addition to applying to MRI images, the proposed method can also be applied to X-ray computed tomography(CT) images, X-ray (X-ray) images, and positron emission computed tomography (PET) images super-resolution through transferring network structures, indicating favorable universality of the proposed method.
In summary, the proposed method is superior to traditional interpolation-based methods and deep learning-based SRCNN, VDSR, SRResNet and SRDenseNet methods.

Conclusion
In order to address the issue that medical image would suffer from severe blurring caused by the lack of highfrequency details in the process of image super-resolution reconstruction, a novel medical image super-resolution method was proposed. Experimental results showed the proposed method is superior to mainstream image super-resolution methods. This work provides a new idea for theoretical studies of medical image super-resolution reconstruction. Fig.4 Comparison of qualitative results of each super resolution method with scaling ratio 2