{'fraction_non_alphanumeric': 0.024358800379970146, 'fraction_numerical': 0.0057674039896865245, 'mean_word_length': 4.593548387096774, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Image Dehazing Algorithm Based On Improved Guided Filtering

Aiming at the problem that the image defogging algorithm based on the dark channel prior will cause the edge halo effect in the restored image, an image defogging algorithm based on improved guided filtering is proposed. First, the calculation method of atmospheric light value based on the quadtree algorithm is adopted to obtain a more accurate atmospheric light value; then, the edge weight factor based on the Canny operator is added to the guided filtering algorithm to solve the local halo effect; finally, the image is restored based on the atmospheric physical model. Experiments show that this algorithm is better than other algorithms in subjective visual effects, and the image edge defogging is more natural; the comparison of objective performance indicators shows that the improved defogging algorithm has higher image quality and better image restoration.


Introduction
In the haze weather or low light, it will greatly affect the image recognition effect. Therefore, the study of defogging algorithms can improve the recognition efficiency. The current defogging methods mainly include two categories. One is the defogging algorithms based on non-physical models, which use image enhancement algorithms to improve image contrast to achieve the defogging effect. The common ones are based on histogram equalization [1] and algorithms based on the principle of Retinex [2,3]. But this method will lose image information; the other is the defogging algorithm based on the physical model, which is mainly based on the classic dark primary defogging algorithm proposed by He et al. [4] to restore the image. This method has a good dehazing effect, but it also has the problem of longer calculation time due to the Soft Matting method. Later, guided filtering was proposed to replace Soft Matting's defogging algorithm [5], but the halo effect will appear after defogging. Some improved algorithms have also been proposed for the above problems. For example, literature [6] uses single-time mean filtering, which has lower algorithm complexity and better dehazing effect. But there will be white fog in the depth of field area; literature [7] proposes a non-local prior algorithm to overcome the dehazing image blocking effect and captures the color distance frame by frame, but the assumption that there is no fog in the distant view may have errors in practice; literature [8] proposed an adaptive transmittance defogging algorithm to solve the problem of inaccurate estimation of the traditional dark channel prior transmittance. But there is a problem of image information loss when judging the fog density using the average gradient value. Based on the dark channel prior model proposed by He et al., this paper improves the atmospheric light value estimation method and the optimization of the transmittance algorithm. The quadtree image segmentation algorithm [9] is used to estimate the atmospheric light value, and when calculating the transmittance, the edge weight factor is introduced into the guided filter algorithm to solve the local halo effect and make the restored image clearer.

Physical model
In the field of machine vision and image research, the mathematical model of fogged images is defined as the following formula [10]: Where I is the original foggy image taken, J is the clear and fog-free image after restoration, t is the light transmittance, and A is the global atmospheric light component.

Dark channel prior defogging algorithm
According to the statistical analysis of more than 5000 pictures by He et al., it is concluded that in most non-sky local areas, some pixels always have a very low color channel (close to 0). Therefore, for any image J , the dark channel can be defined as: is the area centered on pixel x, and dark J is the dark primary color of J .
According to the a priori theory of dark primary colors: According to equation (1) and the dark primary color a priori theory, calculate the minimum value of equation (1) twice, and the estimated value expression of the transmittance ( ) t x  is: Where, ( ) 0 1 ω ω < ≤ is a constant, which is used to retain a certain degree of fog and maintain the depth of the image.
When ( ) ( ) J x t x is close to zero in equation (1), noise will be introduced. If the value of ( ) t x  is too small, the overall image will be excessive to the white field, so the threshold value 0 t is set to obtain the image J after dehazing: The atmospheric light component A is the first 0.1% brightest point in the dark channel, that is, the point with the smallest transmittance, and then at the corresponding point in the dehazing map, the largest value among all the selected channels is selected as A . The calculation of atmospheric light value by He et al. is to find the first 0.1% brightness pixels in the entire dark channel picture pixels as the atmospheric light value [5]. This algorithm has a better recognition effect for natural scene images, but if the image contains non-sky light sources such as lights and white highlight objects, it will cause deviations in the estimation of atmospheric light values and affect the restoration effect of defogging images.

Improved atmospheric light value estimation algorithm
This paper uses the atmospheric light value estimation algorithm [5] based on the quadtree image segmentation algorithm [9] and the He algorithm. The quadtree image segmentation is shown in Figure  1. This algorithm divides the gray image of the smallest channel into four sub-images of the same area, and then calculates the sum of the gray values of each sub-image. Compare and select the best subpicture to continue the decomposition process until the area of the sub-picture reaches the set threshold. Finally, the average value of the pixels with the first 4% brightness in the sub-image is selected as the atmospheric light value 1 A . The specific steps of using the quadtree algorithm to estimate the atmospheric light value are as follows, and the threshold is set to 15×15 pixels.
Step 1 Image quadrate processing. The target image is divided into four sub-images with the same area, and the sum of the gray values of the pixels in the sub-image area is calculated, and the sub-image with the larger sum is selected as the next target image.
Step 2 Calculate the image area. If the target image area is smaller than the set threshold T , the average value of the top 4% of the pixels in the target image area is calculated as the atmospheric light value 1 A . Proceed to step 1.
(a) Original image (b) Grayscale image (c) Image processing diagram Figure 1. Quadtree processing graph.
In order to better estimate the atmospheric light value, what algorithm is used to estimate the atmospheric light estimate 2 A , and the following calculations are performed: The final estimated value A is obtained, and the value range of γ is selected according to experience [0.5, 0.6].

Improved guided filtering
When the guided filtering algorithm filters the image p to be processed, the concept of the guided image I is introduced. The guided image I can be an independent image or the image to be processed. It is assumed that the output image q is the local linearity of the guided image I in the window k area. model: Where k ω is a rectangular area centered on pixel k , and k a and k b are the linear constants of the output image q with respect to the input guide image I .
In order to ensure that the mean square error between the output image q and the input image p is the smallest, the least squares method is used for fitting calculation, which is equivalent to: Where ε is the regularization factor proposed to prevent over-fitting. Improper selection of this value will affect the output image.
For the solution of k a and k b , the calculation is as follows: Among them, k μ and k p respectively represent the mean value of the guide image I . and the input image in the window, and 2 k σ represents the variance value of the guide image I .
Knowing the values of k a and k b , the output image q can be solved by equation (8), but k a and k b in each window are different, and repeated calculations are time-consuming, so the method of calculating the average value first is adopted, as shown below: Where, i a and i b respectively represent the mean value of k a and k b in the k ω window. This algorithmic formula sums the median value of each window and takes the average value, which is faster, less complex and more efficient than the soft matting algorithm. It can be seen from equation (12) that when the guide image I is the same as the input image p , k a can be rewritten as the following equation: It can be seen from the above formula that the value range of k a is [0,1], which is the scale value of the guide map I . When the input image p is selected as the guide image, if ε is large, k a will approach 0 regardless of the value of 2 k σ , which is equivalent to mean filtering; if ε is small, no matter what the value of 2 k σ , k a will tend to Close to 1, there is no filtering effect, and the output image is the input image. The selection of the regularization factor ε in the He algorithm is a fixed value. It does not take into account that the value of the variance 2 k σ in each k ω window is different. In the window area with more image information, there may be spots on the edge of the image, Affect the filtering effect.
For this reason, this article adaptively adjusts ε , rewrites ε as In the formula, σ and G respectively represent the mean square error and mean gradient value in the image area.
(a) Original image (b) Transmittance diagram of guided filter (c) Improved transmittance of guided filter Figure 2. Improved guide filter processing diagram.

Algorithm flow
The algorithm flow chart of this paper is shown in Figure 3.   • Find the minimum value of the three color channels of the image; • Use quadtree algorithm and He algorithm to obtain atmospheric light estimates 1 A and 2 A respectively, and then obtain the final atmospheric light value A ; • Calculate the rough transmittance of the image according to formula (4); • The Canny operator is used to detect the edge area of the image, and the filtering algorithm is improved by the weighting factor η to obtain the fine transmittance; • Restore the image according to formula (8).

Experimental results and analysis
In order to verify the feasibility of the proposed algorithm, this paper conducts experimental comparisons on multiple haze images, and analyzes both subjective evaluation and objective data. Algorithm operating environment: operating system is Windows10, software platform is MATLAB R2017b, computer hardware equipment is Intel(R) Core i7 @2.21 GHZ, system memory is 8GB.

Subjective evaluation
Several typical foggy images are selected for experiments, and the algorithm is compared with the histogram equalization algorithm, the algorithm of literature [6] and the He algorithm for dehazing. The results are shown in Figure 4.  Comparing the above four algorithms, it can be found that although the first three algorithms have some defogging effects, they also have some problems. For example, the overall picture of the picture processed by the histogram equalization algorithm is white and the color saturation is too low. Compared with the original picture, it is found that the distortion is more obvious, and the defogging is unnatural; literature [6] algorithm presents better results than histogram, but blue areas appear when processing sky areas; the algorithm of He et al. has a certain dehazing effect after introducing the guided filtering algorithm, but there is still fog in the edge area of the image. The image restored by the improved defogging algorithm in this paper performs well in the processing of sky area, image saturation, image edge "halo" effect and image distortion rate, and the visual effect is more natural.

Objective comment
The above evaluation is based on subjective evaluation. Next, the algorithm in this paper will be evaluated from the three aspects of algorithm running time, structural similarity index and information entropy from a more accurate and objective perspective. The comparison results are shown in Table 1, Table 2, and Table 3. It can be seen from the above table that the running time of the algorithm in this paper is similar to that of the histogram algorithm. The histogram algorithm is higher than the algorithm in the information entropy, and the algorithm in this paper is much higher than the histogram algorithm in the structural similarity index. This shows that the histogram algorithm restores a higher image quality, but the original image information is lost. The algorithm in this paper retains more detailed information of the original image while ensuring image quality; the algorithm in literature [6] and He et al.'s algorithm are better than the algorithm in this paper in terms of running speed, because the algorithm in literature [6] only uses one-time mean filter on the original image, and the algorithm complexity is low. The algorithm of He et al. has greatly improved the efficiency of the algorithm after changing the soft matting algorithm to the guided filtering algorithm. This article is also based on the guided filtering algorithm, and uses the quadtree algorithm to estimate the atmospheric light value, so the running time is longer. But in terms of structural similarity and information entropy index, the algorithm in this paper is better than both.
Therefore, this paper evaluates the algorithm through the subjective visual effect of the image and the objective information index, and compares it with the histogram algorithm, the algorithm in literature [6] and the algorithm of He et al. It can be seen that the overall operation effect of the algorithm proposed in this paper is the best, and it has better performance in image detail information retention and image quality.

Conclusion
The image restored by the He algorithm has the problem of edge halo effect. In this regard, this article improves the atmospheric light value estimation method, and the introduced quad-tree algorithm can more accurately estimate the atmospheric light value. This paper proposes an improved guided filtering algorithm based on Canny operator, which enhances the edge of the image and retains more image edge information. The algorithm in this paper is compared with the histogram algorithm, the algorithm in [6] and the He algorithm. Experiments show that the algorithm in this paper has a better and more natural dehazing effect on the edge of the restored image, avoids the loss of image edge information, and has higher image quality.