{'fraction_non_alphanumeric': 0.027745104822528194, 'fraction_numerical': 0.011104960907770012, 'mean_word_length': 4.949166495163614, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Multi-stage image denoising with the wavelet transform

Deep convolutional neural networks (CNNs) are used for image denoising via automatically mining accurate structure information. However, most of existing CNNs depend on enlarging depth of designed networks to obtain better denoising performance, which may cause training difficulty. In this paper, we propose a multi-stage image denoising CNN with the wavelet transform (MWDCNN) via three stages, i.e., a dynamic convolutional block (DCB), two cascaded wavelet transform and enhancement blocks (WEBs) and a residual block (RB). DCB uses a dynamic convolution to dynamically adjust parameters of several convolutions for making a tradeoff between denoising performance and computational costs. WEB uses a combination of signal processing technique (i.e., wavelet transformation) and discriminative learning to suppress noise for recovering more detailed information in image denoising. To further remove redundant features, RB is used to refine obtained features for improving denoising effects and reconstruct clean images via improved residual dense architectures. Experimental results show that the proposed MWDCNN outperforms some popular denoising methods in terms of quantitative and qualitative analysis. Codes are available at https://github.com/hellloxiaotian/MWDCNN.


Introduction
Noisy images often arise in the high-level vision tasks, which makes image denoising become an important task in the field of low-level vision [28]. Specifically, a clean image as well as x of image denoising problem can be represented via a degradation model of x = y − n, where y and n stand for a given noisy image and additive white Gaussian noise (AWGN) of standard deviation σ, respectively [28]. According to that, scholars developed a lot of denoisers [32]. For instance, Wang et al. combined a switching scheme and progressive methods based median filter to obtain more detailed information for salt-pepper image denoising [32]. To improve denoising performance, prior knowledge was used to extract useful information in image denoising [24]. Rabbani et al. depended on Laplacian random variables of high local correlation to obtain an estimator based maximum a posteriori and minimum mean squared for image denoising [24]. To improve denoising effect, mapping similar 2D-fragments of corrupted images into 3D-data arrays can enhance sparsity in image denoising [8]. Besides, using different gradient vectors to conduct absolute cosine value can recover detailed information [16]. Also, an edge-preserving regularization term is used to overcome adverse effect of unreliable prior from a guidance image for image denoising [16]. Although mentioned these methods have performed well in image denoising, they suffered from two shortages as follows [35]. That is, they resorted to manually selecting parameters. Also, they may require complex optimization algorithms to pursue excellent denoising effects.
To address the problem, discriminative learning methods in end-to-end ways rather than manually tuning parameters are developed [28]. Due to powerful learning abilities, discriminative learning methods, especially convolutional neural networks (CNNs) are applied to resolve image denoising problem [35]. Zhang et al. exploited combinations of stacked convolutions, batch normalization (BN) and ReLU to implement an efficient CNN denoiser [35]. To obtain salient features, attention idea is embedded into a network in image processing application [29]. For instance, Tian et al. used an attention mechanism to guide a CNN to extract more accurate information for image denoising [29]. To improve denoising performance, making full use of hierarchical information can facilitate more useful information to suppress noise [27]. For instance, Tai et al. utilized recursive units and gate units to transfer memory abilities of shallow layers to deep layers to improve quality of restored images [27]. To further improve performance in image restoration, Zhang et al. used residual dense blocks in a deeper CNN to integrate local and global features for obtaining more robust features in image restoration [39]. Although these methods are effective in image denoising, they may rely on deeper architectures to pursue excellent denoising performance, which may increase difficulty of training. In this paper, we present a multi-stage image denoising CNN with the wavelet transform as well as MWDCNN. It relies on three stages, i.e., a dynamic convolutional block (DCB), two cascaded stacked wavelet transform and enhancement blocks (WEBs) and a residual block (RB). DCB dynamically adjusts parameters of several convolutions via a dynamic convolution to overcome drawbacks of poor performance of some lightweight CNNs in terms of network depth and width for making a tradeoff between denoising performance and computational costs. Stacked WEBs use combinations of signal processing techniques (i.e., wavelet transform) and discriminative learning (i.e., residual dense blocks) to suppress noise for recovering more detailed information in image denoising. To further remove redundant features, RB is used to refine obtained features for improving denoising effects and reconstruct clean images via improved stacked residual dense architectures. Besides, the proposed MWDCNN is superior to popular denoising methods, i.e., a denoising CNN (DnCNN) [35] and attention-guided denoising (ADNet) [29] in terms of quantitative and qualitative analysis.
Contributions of this paper can be shown as follows.
(1) A dynamic convolution is used into a CNN to address limitations in depth and width of lightweight CNNs for pursuing good denoising performance.
(2) The combination of a signal processing technique and discriminative learning technique is used for image denoising.
(3) Enhanced residual dense architectures are used to remove redundant information for improving denoising effects.
The remainder of this paper is organized as follows. Section 2 provides related work about deep CNNs for image denoising, dynamic convolutions and wavelet transform techniques. Section 3 describes the proposed denoising method. Section 4 shows datasets, experimental settings, the proposed method analysis and extensive experimental results. Section 5 reports conclusion of this paper.

Deep CNNs for image denoising
To overcome drawbacks of traditional machine learning in image denoising, networks based components, i.e., CNNs are proposed [36]. To accelerate training speed of denoisers, Zhang et al.
used patches of noisy images and noisy mapping to act a CNN to achieve an efficient denoising network [36]. To improve denoising performance, combining dilated convolutions and BN in a CNN can obtain more accurate context information to promote denoising effect [28]. Mentioned methods have obtained excellent denoising performance, however, convolutional kernels of these methods have same weights for noisy images, which may have big computational costs. Differing from these methods, our denoising method uses dynamic convolutions in a CNN to adjust parameters of convolutional kernels to train a robust denoiser, where mentioned dynamic convolutions can be shown in Section 2.2. Besides, we combine frequency features via wavelet transform and structural information via a CNN to obtain complementary information in image denoising, where mentioned wavelet transform can be shown in Section 2.3.

Dynamic convolutions for image applications
Most of existing methods share parameters of each convolutional layer to train a denoising CNN for image applications (i.e., image classification), however, they cannot adjust parameters of each layer to obtain a robust classifier, according to different images [5]. To address this question, dynamic convolutions used multiple parallel convolutions via attention mechanisms are fused to reduce computational costs of different CNNs for training a robust classifier [5]. To improve training efficiency, a dynamic convolution was embedded into a CNN to remove redundancy features for improving classification accuracy [40]. To mine more context, Sun et al. used Gaussian dynamic pyramid pooling in a CNN to improve expressive abilities in image segmentation [26].
Besides, using matrix decomposition to guide a dynamic convolution can make a tradeoff between performance and training speed in image recognition [17]. Due to excellent performance, a dynamic convolution is used into a CNN for image denoising in this paper.

Wavelet transform for image applications
It is known that images can be treated as signals, thus, signal processing techniques, i.e., wavelet transform are effective for low-level tasks [7]. For instance, Cho et al. used multivariate statistical idea to estimate coefficients of wavelet transform for filtering noise in image denoising [7]. Liu et al. utilized wavelet transform and genetic algorithm to suppress noise in image denoising [19]. To mine more useful information, wavelet transform is used into a CNN to learn detailed information and content information for image super-resolution [12]. As an alternative, using cross-connection and residual technique to integrate a CNN and wavelet transform was a good tool for image super-resolution [34]. According to mentioned illustrations, we can see that wavelet transform is effective for low-level task. Inspired by that, we fuse two cascaded wavelet transform techniques into a CNN to implement fusion of frequency information and struct information for promoting visual denoising effects.

Network architecture
The designed 23-layer MWDCNN contains three parts, i.e., a DCB, two cascaded WEBs and a RB for image denoising as shown in Fig.1  noise. That is, it combines frequency features via wavelet transform and structure information via convolutional layers to mine robust features in image denoising. Also, a 10-layer RB is used to refine obtained features via enhanced residual dense architectures and reconstruct clean images via a residual learning operation. To vividly express the mentioned process, the following equation is conducted.
where I N and I C denote given a noisy image and a clean image, respectively.  [13]. Besides, to show its good performance of MSE, we discuss validity of different losses from Charbonnier of the newest denoising method [18] and Pearson of quality assessment method [3] fused into our MWDCNN for image denoising in this Section 4.3. The mentioned procedure can be formulated as Eq. (2).
where l and θ express loss function and parameter sets.

Dynamic convolution block
The first stage of MWDCNN is implemented by a dynamic convolution block. The dynamic convolution block is composed of five layers: a convolutional layer, a 3-layer dynamic convolution and a convolutional layer. The first convolutional layer is used to convert noisy images to linear features, where its parameters are input channel number of 3 or 1 (depending on whether the input is a color or gray noisy images), convolutional kernel of 5 × 5 and output channel number of 64.
The 3-layer dynamic convolution consists of 2-layer weight generator and 1-layer convolutional layer with 5 × 5 , where the weight generator contains an average pooling, a combination of convolutional layer with 1 × 1 and ReLU, a convolutional layer with 1 × 1, and Softmax as illustrated in Fig. 2 [5]. The second convolutional layer followed by ReLU is used to refine obtained features from the dynamic convolution, where its parameters are input channel number of 64, convolutional kernel of 5 × 5 and output channel number of 64. These illustrations can be presented as follows.
where DC denotes a function of dynamic convolution and O DCB is output of DCB, which acts two stacked WEBs. C, R and CR stand for a convolution operation of 5 × 5, activation function as well as ReLU and convolution operation of 5 × 5 with ReLU, respectively.
Implementations of mentioned dynamic convolution are shown as follows [5]. Firstly, it uses WG to obtain four weights to act four parallel convolutional kernels in a weighted way to adjust parameters. Secondly, obtained results act a convolutional layer of 5 × 5 . Thirdly, output of the first convolution in a DCB and output of second step in a dynamic convolution linearly fuse obtained features by obtaining suitable parameters for different noisy images. Work of a dynamic convolution in a DCB can be symbolled by the following Equations.
where O DCB 1C is an output of the first convolutional layer in the DCB. DC denotes function output of a dynamic convolution. Specifically, a function of WG [5] can be defined as Eq. (5).
where AP and S are defined as an average pooling and Softmax, respectively. RC 1 and C 1 are the combination of a convolutional layer of 1 × 1 and a ReLU, and a convolutional layer of 1 × 1.

Two stacked wavelet transform and enhancement blocks
The second phase of WDMCNN uses two stacked wavelet transform and enhancement blocks.
Specifically, each 4-layer wavelet transform and enhancement block as well as WEB includes three steps to fuse frequency features and structure information via combining signal processing techniques and discriminative learning for obtaining more useful information. The first step uses discrete wavelet transform (DWT) [9] to convert linear construct information to four frequency features. The second step utilizes structural network to guide signal processing techniques via a feature enhancement (FE) mechanism (also treated as a 4-layer residual dense block (RDB) in Fig. 3 [39]) to suppress noise for recovering more detailed information in image denoising. It is noted that sizes of all convolutional kernels besides the last covolutional kernel in a FE are 5 × 5 and the last convolutional kernel in a FE is 1 × 1. The third step uses an inverse discrete wavelet transform (IDWT) to convert frequency features to linear structure information. The mentioned illustrations can be shown in Eq. (6) O where f W EB expresses a function of WEB. f DW T and f IDW T denote functions of DWT and IDWT, respectively. f F E and O W EB represent function of FE and output of two stacked WEBs, respectively. More information of mentioned DWT and IDWT can be obtained in Ref. [9]. Also, mentioned 4-layer FE as well as a RDB contains three combinations of a convolutional layer and ReLU (also treaded as Conv+ReLU) and a convolutional layer (also treaded as Conv with 1 × 1) and its more detailed information can be obtained in Ref. [39].

Residual block
The RB is implemented a 10-layer enhanced residual dense architecture. That is, it is composed of two combinations of residual dense block and ReLU and two convolutional layers as well as RDC, and two residual learning operations. That can be implemented by four steps as follows.
The first step uses two combinations of a 4-layer RDBs [39] and ReLU as the first component to further refine features in image denoising. To prevent long-term dependency problem, the second step uses two residual learning operations to fuse obtained features from two combinations of each RDB and ReLU, and WEB to enhance memory abilities of shallow layers on deep layers in image denoising. To prevent over-enhancement phenomenon of the second step, the third step uses a convolutional layer followed by ReLU to refine these features, where its input channel and output channel are 64, respectively. Finally, a convolutional layer is used to reconstruct noisy mappings and a residual learning operation is used to reconstruct clean images via obtained noisy mappings and given noisy images. izes of all convolutional kernels besides the fouth and eighth convolutions in a residual block are 5 × 5. Also, the fourth and eighth convolutional kernels are 1 × 1. Input channel number and output channel number of two RDBs are 64, and input channel number of the last convolutional layer is 64. And the output channel number of the last convolutional layer is 3 or 1, depending on whether the noisy image is a color or gray images. The illustrations above can be shown in Eq. (7). Figure 3: Architecture of feature enhancement (FE) as well as residual dense block (RDB).
where f RDC denotes a function of RDC. Also, '-' stands for a residual learning operation as well as ⊕ in Fig. 1. f RDB and R express a function of RDB and ReLU, respectively.  [33]. Also, we cut each image in real noisy image dataset as 211,600 image patches of augmentation with eight ways to make training images richer, mentioned eight ways can be shown as Ref. [29].

Network analysis
In this paper, we analyze designed denoising network to show its rationality and validity as follows.
Dynamic convolutional block: Robust denoisers are very important for different scenes. However, most of existing denoising methods resorted to sharing parameters to train denoising models, which cannot make these denoising models robust for different scenes. That also has huge challenges for high-level vision tasks. Also, dynamic convolution can adjust parameters of convolutional kernels, according to given images [5]. Inspired by that, we fuse a dynamic convolution into a CNN to suppress noise for improving denoising performance without increasing computational costs. That is, we design a dynamic convolutional block as well as DCB to address the mentioned problems. DCB uses a convolutional layer to extract linear features. Subsequently, a dynamic convolutional layer [5] is used in a CNN to linearly combine all convolutional kernels to dynamically adjust parameters rather than same parameters of convolutions, according to different noisy images, which can make a tradeoff between denoising performance and computational costs as follows.
In terms of denoising performance, we design three experiments to verify denoising effect  Table 1, we can see that the combination of three stacked convolutional layers and a dynamic convolutional layer obtains higher value than that of combination of six stacked convolutional layers, which tests effectiveness of individual for image denoising.
According to mentioned analysis, it is known that a dynamic convolution in a CNN is effective for image denoising.
In terms of computational costs, we choose important metrices, i.e., complexities (parameters and Flops) and run-time to show denoising competitiveness of a dynamic convolution on real digital cameras in the real world. That is, the combination of three stacked convolutional layers and a dynamic convolutional layer obtains competitive both parameters and flops than that of combination of six stacked convolutional layers as illustrated in Table 2, which shows competitive complexities of a dynamic convolution for image denoising. The combination of three stacked convolutional layers and a dynamic convolutional layer obtains less run-time than that of combination of six stacked convolutional layers in Table 3, which shows fast execution denoising ability of a dynamic convolutional layer. Thus, a dynamic convolutional layer used in a CNN is very reasonable and valid for image denoising in this paper. Besides, we use a convolutional layer in a DCB to extract useful information, where its effectiveness is tested via MWDCNN and MWDCNN without a convolution in a DCB in Table 1.
Wavelet transform enhancement block: It is known that images can be treated as signals, thus, signal processing techniques, i.e., wavelet transform are effective for low-level tasks [7]. Be-

Methods
Parameters Flops The combination of six stacked convolutional layers 0.212M 5.59G The combination of three stacked convolutional layers and a dynamic convolutional layer 0.498M 2.81G

Methods
Run-time The combination of six stacked convolutional layers 0.060s The combination of three stacked convolutional layers and a dynamic convolutional layer 0.046s Residual block: According to Ref. [1], we can see that stacked convolutional layers can refine obtained features. To enhance robustness of obtained denoiser and prevent dependency of deep networks, we propose a 10-layer residual block as well as RB via improved residual dense architectures to improve quality of predicted images and construct images as follows. Firstly, we use two stacked RDBs (each RDB with three convolution kernels of 5 × 5) to refine obtained features from stacked WEBs, its effectiveness is tested by MWDCNN with a convolutional layer and without DCB, two WEBs, two ReLUs and two RLOs in RDC and MWDCNN with a convolutional layer and without DCB, two WEBs, two ReLUs, two RLOs and a RDB in RDC in Table 1. Secondly, ReLU acts each RDB to convert obtained features into non-linearity, where its effectiveness is verified through MWDCNN with a convolutional layer and without DCB and two WEBs and MWDCNN with a convolutional layer and without DCB, two WEBs and two ReLUs in RDC as presented in Table 1. To address long-term dependency of deep networks, we use two residual operations to fuse obtained features from the second combination of RDB and ReLU and the second WEB. MWDCNN with a convolutional layer and without DCB, two WEBs and two ReLUs in RDC outperforms MWDCNN with a convolutional layer and without DCB, two WEBs, two Re-LUs and two RLOs in RDC in PSNR as listed in Table 1, which shows its effectiveness. Fourthly, we use a convolutional layer of 5 × 5 to prevent over-enhancement phenomenon from the previous steps. MWDCNN with a convolutional layer and without DCB, two WEBs, two ReLUs, two RLOs and a RDB in RDC has obtained higher PSNR value than that of MWDCNN with a convolutional layer and without DCB, two WEBs, two ReLUs, two RLOs, a RDB and a convolutional layer in RDC to test effectiveness of a convolutional layer. According to mention illustrations, the designed block is reasonable and valid for image denoising. Also, MWDCNN has obtained better denoising performance than that of MWDCNN with a convolutional layer and without an RDC for image denoising in Table 1, which verifies effectiveness of RDC for MWDCNN in image denoising. Besides, ReLUs as important components of MWDCNN are used to convert linear data into non-linearity, where its effectiveness is verified by MWDCNN and MWDCNN without ReLUs in Table 1.
In terms of choosing loss function, we take into the following three aspects consideration. The first aspect utilizes a loss function as well as MSE of popular denoising methods as an objective function in terms of previous denoising work. The second aspect uses a loss function (Charbonnier) of the newest denoising method based Transformer to fuse into the proposed MWDCNN to test denoising performance in terms of the newest denoising work. The third aspect exploits typical quality assessment method of Pearson as a loss function to test denoising performance in terms of perceptual theory. As shown in Table 4, we can see that the proposed method with MSE is superior to the proposed method with Charbonnier, which shows MSE is suitable to lightweight denoising CNN and Charbonnier [18] is useful for big denoising CNN, i.e., Transformer. Also, only using MSE as a loss function outperforms using the combination of MSE and Pearson as a loss function [3] in terms of PSNR and SSIM for image denoising, which shows fusion of two different losses may cause a negative effect. Thus, we choose MSE as a loss function to train a MWDCNN denoiser.
In a summary, the dynamic convolutional block with a dynamic convolution is strong adaptability for different scenes. Also, it is very competitive in terms of denoising performance and computational costs. To further improve denoising performance, a wavelet transform enhancement block relies on a combination of a single processing technique and discriminative learning technique via fusing wavelet transform technique and a CNN is designed. To improve the robust-ness of obtained MWDCNN denoising model, a residual block uses a stacked architecture to refine obtained information. According to mentioned illusrations, we can see that key components, i.e., dynamic convolutional block, wavelet transform enhancement block, residual block and chosen loss function are reasonable and valid for image denoising, which makes MWDCNN reasonable and valid in image denoising in this paper.        Noise Level σ = 15 σ = 25 σ = 35 σ = 50 σ = 75 DnCNN-B [35] 0.9205 0.8774 0.8394 0.7896 --ADNet [29] 0.9239 0.8820 0.8445 0.7984 0.7387 FFDNet [36] 0
In  result in PSNR on CBSD68 for mentioned five noise levels in Table 7. Also, our MWDCNN is more effective than these of other comparative denoising methods on Kodak24 for five noise levels above in Table 8. These illustrations our MWDCNN is a good tool for color Gaussian noisy image denoising.
For real noisy image denoising, we use CC to test denoising performance of different methods for real digital devices in the real world. As shown in  [25] to conduct experiments as shown in Table 14, which shows our methods is more effective than DnCNN-B, ADNet and FFDNet in low-and high-noise levels. The best denoising results are marked by red and blue lines from Table 5 to Table 14..
Due to difficult improvements of denoising methods, improvement of 0.01dB from denoisers are very reasonable [28,35]. Thus, although some MWDCNN have only a little improvements of PSNR than that of the second image denoising methods for certain noise level for gray and color Gaussian noisy image denoising, they are very competitive for image denoising. Also, the proposed MWDCNN is very effective for real noisy image denoising. Besides, our MWDCNN is beneficial to low-level metrics, i.e., FSIM, SSIM and LPIPS and high-level metric, i.e., Inception-Score. These illustrate the proposed MWDCNN is effective and robust in image denoising in terms of quantitative analysis.
In terms of qualitatively analysis, we choose popular denoising methods, i.e., FFDNet, ADNet, DnCNN and our MWDCNN to obtain predicted images to compare visual denoising performance.
The predicted images of different methods can be chosen one area to amplify this area as observation area, the mentioned observation area is clearer, its corresponding denoising method is more effective for image denoising. For gray noisy image denoising, we choose two images from the Set12 to conduct visual experiments for noise levels of 15 and 25, respectively. As shown in Fig.   4 and Fig. 5, we can see that our MWDCNN is clearer than these of other denoising methods in the observation areas. That shows it is more effective in visual perspective for gray image denoising. For color noisy image denoising, we choose two images from the Kodak24 to conduct visual experiments for noise levels of 25 and 35, respectively. As shown in Fig. 6 and Fig. 7, we can see that our MWDCNN is clearer than other denoising methods in the observation areas. That illustrates it is more effective in perspective for color image denoising. According to mentioned descriptions, we can see that the proposed MWDCNN is more superior than other popular denoising methods in qualitatively analysis. In a summary, our MWDCNN is a good choice for image denoising, according to quantitative and qualitative analysis.
According to analysis of Sections 4.3 and 4.4, we can see that due to use of a dynamic convolution, the proposed MWDCNN is strong adaptability and computational costs for different scenes.
Due to combination of a single processing technique and discriminative learning technique as well as fusing wavelet transform technique and a CNN, the proposed MWDCNN has better denoising performance. Due to use of stacked architectures, our MWDCNN is robust for image denoising.
Thus, the proposed MWDCNN is effective to complex scenes for portable digital devices, i.e., smartphones and cameras.

Conclusion
In this paper, we propose a multi-stage image denoising CNN with the wavelet transform as well as MWDCNN in image denoising. The first stage of MWDCNN uses linearly combination of several convolutional kernels to dynamically adjust parameters rather than same parameters of convolutions, according to different noisy images, which can make a tradeoff between denoising performance and computational cost. It is very useful for noisy images with unknown noise. The second stage of MWDCNN fuses frequency features and structure features via two combinations of wavelet transformation and residual dense block to suppress noise, which enhances robustness of obtained denoiser for complex scenes. The third stage of MWDCNN uses improved residual dense architectures to remove redundant features to improve performance of image denoising. The proposed method is very beneficial to damaged images with unknown noise. However, it depends on a supervised manner to train a denoising model. Also, collected noisy images by cameras are difficult to obtain clean reference images. Thus, we will conduct clean reference images to train a blind denoiser in the future, according to properties of noisy images.