{'fraction_non_alphanumeric': 0.03605351170568562, 'fraction_numerical': 0.01157190635451505, 'mean_word_length': 4.533308660251666, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Image Denoising using Optimally Weighted Bilateral Filters: A Sure and Fast Approach

The bilateral filter is known to be quite effective in denoising images corrupted with small dosages of additive Gaussian noise. The denoising performance of the filter, however, is known to degrade quickly with the increase in noise level. Several adaptations of the filter have been proposed in the literature to address this shortcoming, but often at a substantial computational overhead. In this paper, we report a simple pre-processing step that can substantially improve the denoising performance of the bilateral filter, at almost no additional cost. The modified filter is designed to be robust at large noise levels, and often tends to perform poorly below a certain noise threshold. To get the best of the original and the modified filter, we propose to combine them in a weighted fashion, where the weights are chosen to minimize (a surrogate of) the oracle mean-squared-error (MSE). The optimally-weighted filter is thus guaranteed to perform better than either of the component filters in terms of the MSE, at all noise levels. We also provide a fast algorithm for the weighted filtering. Visual and quantitative denoising results on standard test images are reported which demonstrate that the improvement over the original filter is significant both visually and in terms of PSNR. Moreover, the denoising performance of the optimally-weighted bilateral filter is competitive with the computation-intensive non-local means filter.


INTRODUCTION
We consider the standard problem of denoising grayscale images that are corrupted with additive white Gaussian noise [1,2,3]. In this setup, we are given the corrupted (or noisy) image where I is some finite rectangular domain of Z 2 , {f0(ı) : ı ∈ I} is the unknown clean image, {wı : ı ∈ I} are independent and distributed as N (0, 1), and σ is the noise level. The goal is find a denoised estimatef (ı) of the clean image from the corrupted samples. The denoised image should visually resemble the clean image. One way to quantify the resemblance is using the mean-squared-error (MSE) which is defined to be Later in the paper, we will use the peak signal-to-noise ratio (PSNR) which is given by PSNR = 10 log 10 (255 2 /MSE). The image denoising problem has been extensively studied and a survey of even a fraction of this literature is beyond the scope of this paper. We instead refer the interested reader to [1] - [12] and the references therein.
Our present interest is in the image denoising applications of the edge-preserving bilateral filter [13,14,15]. The denoised image in this case is set to bê .
The Gaussian kernels in (4) are respectively referred to as the spatial and range kernels. In practice, the domain Ω is restricted to some neighbourhood of the origin. Typically, Ω is a square neighbourhood, [13]. We refer the reader to [13,16] for a detailed exposition on the working and in particular the edge-preserving property of the filter. The bilateral filter has received renewed attention in the image processing community in the context of image denoising [17,18]. It is well-known that, while the filter is quite effective in removing modest levels of additive noise, its denoising performance drops at large noise levels [16]. It was demonstrated in [6,7] that a patch-based extension of the filter can be used to bring the denoising performance of the filter at par with state-of-the-art methods. These, and other advanced patch-based methods [8,9,10], are however much more computation-intensive than the bilateral filter.

Present Contribution
In this work, we present a couple of ideas for improving the denoising performance of the classical bilateral filter, and give fast algorithms for the same. The contributions (and the organization) of the paper are as follows: • In Section 2, we demonstrate how the denoising performance of the bilateral filter can be improved at large noise levels (at almost no additional cost) by incorporating a simple pre-processing step into the framework.
• The modified filter is designed to be robust at large noise levels, and tends to perform poorly below a certain noise threshold. To get the best of the either filters at all noise levels, we propose to optimally combine them in a weighted fashion in Section 3. The optimality is in terms of a certain surrogate of the mean-squared-error (MSE) known as Stein's unbiased risk estimate (SURE) [19]. This MSE-estimate is known to be quite accurate in practice [11,20,21] and, as a result, the combined filter is almost always guaranteed to provide a lower MSE than the original filter.
• Following [22,23], we present an approximation for the proposed filtering in Section 4 that has a fast implementation. We derive the SURE estimator for this approximation, and demonstrate how it can be efficiently computed in the process of approximating the bilateral filter. The overall cost of computing the estimator and the optimally-weighted filters is about twice the cost of computing a single bilateral filtering using the fast algorithm in [22].
We provide both visual and quantitative denoising results on standard test images in Section 5 which demonstrate that the improvement over the original filter is significant both visually and in terms of PSNR.

ROBUST BILATERAL FILTER
Notice that the range filter in (3) operates on the noisy samples. In other words, the corrupted image is used not just for the averaging but also to control the blurring via the range filter. What if the range filter could directly operate on the clean image? It can be verified that the denoising image obtained using this "oracle" filter is visibly better and has higher PSNR, which is not surprising. Of course, we do not have access to the clean image in practice, and thus the oracle bilateral filter cannot be realized. Nevertheless, one could consider some proxy for the clean image. Our proposal is simply to use the box-filtered version of the noisy image as a proxy. In particular, we propose the following robust bilateral filter (RBF): . (5) wheref The amount of smoothing induced by the box filter is controlled by L. We performed exhaustive simulations in this direction. The simulation results suggest that L = 1 (3 × 3 blur) is optimal for most settings. A possible way to explain this is that this small blur is able to suppress the noise without excessively blurring the image features. The denoising results obtained using the standard and the robust filter a couple of test images are shown in Table 1. We note that the filters have been independently tuned with respect to σs and σr to get the best PSNR. These results (and the results on other test images that are not shown here) suggest that the robust filter starts to perform better beyond a certain noise level depending on the type of image. This is not surprising since, at low noise levels, the box filtering introduces more blurring than noise suppression which brings down the overall signal-to-noise ratio. Indeed, the corrupted image is already a good proxy for the clean image when the noise floor is small. On the other hand, notice that the improvement in PSNR is often as large as 10 dB at large noise levels.

OPTIMALLY WEIGHTED BILATERAL FILTERS
The results in Table 1 suggest that we could possibly perform better denoising by combining (3) and (5). A particularly simple idea would be to take a linear combination of these estimates. That is, we could set the denoised image to bê Notice that this includes (3) and (5) as special cases. This bring us to the question of setting the weights θ1 and θ2. This can hypothetically be done by minimizing the MSE between (7) and the clean image. However, we do not have access to the clean image. This is precisely where the classical Stein's unbiased risk estimate (SURE) [19] is useful. This is given by SURE has the property that its expected value equals that of (2) for the Gaussian noise model in (1) [19]. This makes it an useful surrogate for the MSE, which can be computed without the knowledge of the clean image. We note that the idea of taking linear (or affine) combinations of estimators and tuning the weights to get the optimal SURE has been tried in different contexts in the literature; e.g., see [11]. Note that the added computation required for SURE are the computation of the partial derivatives in (8).
We propose to find the optimal weights in (7) by minimizing SURE. In particular, on substituting (7) in (8), we see that SURE is quadratic in θ1 and θ2. Thus, by convexity, a necessary and sufficient condition for optimality is that that the gradient of (8) must vanish at the optimal weights. In particular, it can be verified that the resulting gradient equations can be written as and and θ # = (θ # 1 , θ # 2 ) are the optimal weights. To simply notations, we will henceforth use the shorthand ∂ı in place of the operator ∂/∂f (ı) appearing in (8). In summary, by computing A and b, and solving the 2 × 2 linear equation Aθ # = b, we get the optimal weights. The weights are then plugged into (7) to get the final denoised image.

FAST AND SURE IMPLEMENTATION
The cost of computing (3) and (5) is clearly O(σ 2 s ) per pixel, since the support of the spatial filter is proportional to σ 2 s . Moreover, we are also required to compute the derivatives for SURE which would further add to the cost. We now explain how we can compute (3), (5), and the derivatives ∂ıf1(ı) and ∂ıf2(ı) using the fast algorithm in [22,23]. It was observed here that, for sufficiently large N , we can accurately approximate the range kernel in (4) using where ı denotes √ −1, Plugging (11) into (3), using the multiplication-addition property of exponentials, and exchanging the summations, the numerator of (3) can be expressed as where Hn(ı) = exp(−ıωnf (ı)), Fn(ı) = H n (ı)f (ı), andFn(ı) denotes the Gaussian filtering of Fn: Here and later, we use z to denote the complex conjugate of z.
Similarly, the denominator of (3) can be expressed as where Gn(ı) = H n (ı) andḠn(ı) is as defined in (13). In summary, we can approximate (3) usinĝ We note that the same notation has been used for both (3) and its approximation (14). The computation of (14) is clearly dominated by the computation of a series of Gaussian filtering. Now, each of these filtering can implemented in constant-time (for any arbitrary σs) using separability and recursions [24]. We have thus been able to cut down the per-pixel complexity from O(σ 2 s ) to O(1) using (14). On the other hand, note that we can write After some calculation and simplification, we can verify that and To arrive at the above formulas, we have use the fact that the sum of (cn) is 1 and that of (cnωn) is 0. These identities are obtained by evaluating the both sides of (11) and its derivative at t = 0. The important point here is that some of the quantities that are used for computing (14) are reused to compute the partial derivatives in (15) and (16). The steps of the computation are summarized in Algorithm 1. It is clear that the main computations are the Gaussian filtering in steps 12 and 13. That is, the overall cost is dominated by the cost of computing 2(N + 1) Gaussian filtering. Notice that we have recursively computed the binomial coefficients in (12), and we have introduced temporary variables to cut down some of the redundant computations.

EXPERIMENTS
We now present some denoising results on standard test images. Our objective is to compare the denoising results obtained using the proposed modification with the standard bilateral filter, both visually and in terms of PSNR. In this work, we assume that σ is provided; this has to be estimated in practice from the noisy image. In this regard, we note that it has been demonstrated in [20,21] that the SURE is quite robust to standard data-based estimates of σ. We remark that one can optimize the SURE for the proposed filter with respect to σs and σr as done in [20,21]; however, we do not investigate this possibility in this paper. In Table 2, the denoising performance of the proposed weighted filter at different noise level is compared with the standard and the robust bilateral filter, as well the patch-based non-local means filter [6]. We have used L = 1 in (6) for all the experiments. To have a fair comparison for a given image and noise level, we independently tuned σs and σr to optimize the PSNRs of the respective filters. We also tuned the parameters of NLM to get the optimal PSNR at each noise level. As remarked earlier, the robust filter starts to perform better than the standard filter beyond a certain noise level (σ ≈ 20). Notice that the PSNR obtained using the optimally-weighted filters is consistently higher than that of the constituent filters at all noise levels; the improvement is often as large as 1 dB. On the other hand, the improvement in PSNR over the standard bilateral filter is often as high as 10 dB. This does not come as a surprise since it is well-  Denoising results using standard bilateral filter (SBF) and the proposed weighted bilateral filter (WBF). For reproducibility, we used the builtin Camerman image that comes with Matlab. We tuned the parameters of SBF and WBF to get the optimal PSNR in either case. The optimal (σs, σr) setting is indicated in the caption.
known that the bilateral filter has a lot of scope of improvement at high-noise levels [16]. However, what does come as a surprise is that proposed filter is competitive with the non-local means filter [6] that uses patches (groups of neighboring pixels) instead of single pixels for denoising. For a visual comparison, the results of a particular denoising experiment are shown in Figure 1. Notice that the denoised image obtained using the proposed filter looks much more sharp than that obtained using the standard filter, and has a significantly higher PSNR. Also, notice that the noise in the background is much less in (d) compared to (c). In Table 3, we compare the run times of the direct and the fast implementation of the proposed filter for different parameter settings. Notice that the fast implementation is significantly faster than the direct implementation.

CONCLUSION
We demonstrated how a simple pre-processing step can substantially improve the denoising performance of the bilateral filter at large noise levels. To consistently get the best of the standard and the pre-processed filter at all noise levels, we proposed to optimally weight them using the accurate Stein's unbiased estimate of the MSE. The optimal weights were efficiently found by solving a small linear system. A fast algorithm for implementing the optimally-weighted filters was also described. We reported visual and PSNR results on test images which confirmed the improvement over the original bilateral filter. An interesting finding was that the weighted bilateral filter is competitive with the non-local means filter.