{'fraction_non_alphanumeric': 0.029687281539823834, 'fraction_numerical': 0.00680430628699259, 'mean_word_length': 4.752815013404826, 'pattern_counts': {'":': 0, '<': 6, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 5, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Auto robust relative radiometric normalization via latent change noise modelling

Relative radiometric normalization(RRN) of different satellite images of the same terrain is necessary for change detection, object classification/segmentation, and map-making tasks. However, traditional RRN models are not robust, disturbing by object change, and RRN models precisely considering object change can not robustly obtain the no-change set. This paper proposes auto robust relative radiometric normalization methods via latent change noise modeling. They utilize the prior knowledge that no change points possess small-scale noise under relative radiometric normalization and that change points possess large-scale radiometric noise after radiometric normalization, combining the stochastic expectation maximization method to quickly and robustly extract the no-change set to learn the relative radiometric normalization mapping functions. This makes our model theoretically grounded regarding the probabilistic theory and mathematics deduction. Specifically, when we select histogram matching as the relative radiometric normalization learning scheme integrating with the mixture of Gaussian noise(HM-RRN-MoG), the HM-RRN-MoG model achieves the best performance. Our model possesses the ability to robustly against clouds/fogs/changes. Our method naturally generates a robust evaluation indicator for RRN that is the no-change set root mean square error. We apply the HM-RRN-MoG model to the latter vegetation/water change detection task, which reduces the radiometric contrast and NDVI/NDWI differences on the no-change set, generates consistent and comparable results. We utilize the no-change set into the building change detection task, efficiently reducing the pseudo-change and boosting the precision.


Introduction
The radiance/color information are critic for semantic segmentation of different objects, classification of vegetations, change detection, mapping and retrieval in remote sensing tasks. However, due to the imaging time, light source, solar altitude angle, atmospheric attenuation, sensor type, and other factors, there are inconsistencies in an invariant object's radiances in different remote sensing images. These inconsistencies may result in different reflectance and spectral indices(such as NDVI, NDWI) even after coarse radiometric calibration and atmospheric correction, hindering many tasks, including classifying vegetation and other change detection tasks. Moreover, when mosaicking multiple remote sensing images, these inconsistencies cause radiometric/color contrast in the mosaic image, creating unsightly seam lines (Liu et al. (2021)). Therefore, eliminating or alleviating these inconsistencies becomes essential.
There are two main approaches to dealing with radiometric inconsistencies: radiometric calibration combining with atmospheric correction and relative radiometric correction. The first approach usually utilizes the gain and bias parameters of the satellite sensor to convert digital number to spectral radiance to achieve the radiometric calibration and utilizes atmospheric radiative transfer model,such as 6S (Vermote et al. (1997)), MORTRAN etc, with atmospheric parameters, such as atmospheric optical depth(AOD) obtained from MODIS/MERRA2 data and other satellite physical parameters to implement atmosphere correction. This method is an idealistic way to obtain the abso-lute surface reflectance. Nevertheless, the radiometric calibration step might be subject to the low update frequency of the gain and bias parameter measured by the ground-based calibration. The change range of the parameters can annually be up to 100 percent in some satellite sensors, making estimating these parameters difficult. Besides, the atmospheric correction step might be subject to spatial and temporal resolution of atmospheric parameters. MODIS's spatial resolution ranges from 10km, and the temporal resolution is about one day. Therefore, there might be a lot of precision error induced by the time/spatial resolution. In some extreme cases where gain, bias, or AOD parameters are unavailable, the radiometric calibration and atmospheric correction become intractable. The second approach does not depend on those physic parameters but utilizing a reference image to implement relative radiometric normalization(RRN) to calibrate the radiometric level of the target image to the radio- (1995)) but the main situation is that simple automatic RRN models are not robust disturbing by object change and RRN models precisely considering object change might not either robustly obtain sufficient no-change set. • Image regression models: • Radiometric control sets models: Radiometric control sets methods utilize the sets of scene elements with a mean reflectance remaining nearly unchanged over time (Hall et al. (1991)). Typically, they derive a bright control set using the approach similarly with PIF methods, and a dark control set, which can be the dark water, from the scattergram to linear modeling the relative radiometric normalization mapping. However, the bright radiometric control sets face the same problems described in the PIF methods. Also, nearly unchanged the mean reflectance of the control sets assumption may help linear normalization but maybe not robust in nonlinear cases, particularly when the sub-object reflectance in the control set changed.
• No-change set model: No-change set methods (Elvidge et al. (1995)) utilize the infrared band and red band scattergrams to determine the water and land maximum radiance value among the source and target images to establish a coarse relative radiometric normalization mapping of the infrared band and red band. They use the threshold of normalizing error of coarse RRN mapping to determine the no-change set and use the no-change set to determine the fine RRN mapping. These methods are pretty novel since they utilize the normalization error to determined the no-change set. However, water may not always exist in the image, which is fatal for these methods' robustness. Some minor issues of these methods could be the nonlinearity of the radiometric inconsistency and the disuse of the image's red and green band. Sometimes, this method might need human intervention to obtain stability. One promising direction is to RRN that is the no-change set root mean square error. We apply the HM-RRN-MoG model to the latter vegetation/water change detection task, which reduces the radiometric contrast and NDVI/NDWI(McFeeters (1996)) differences, generates consistent and comparable results. We utilize the no-change set into the building change detection task, efficiently eliminating the pseudo-change and boosting the precision. In conclusion, our method has advantages as follows: • automatic and robust: it automatically robustly extracts no-change set with all band information(robust to clouds/fogs/changes).
• evaluable and accurate: it is an accurate relative radiometric normalization using histogram matching and can be evaluated through robust no-change set mean square error and log-likelihood.
• probability and mathematics theoretically grounded: it is under probability model theoretical support and bridges of the histogram matching model with the L1-norm image regression RRN model with monotone mapping.
• useful for downstream tasks including vegetation/water cover change detection, object change detection task(eliminating pseudo-change).

Methodology
Our method is based on discovering the no-change set and simultaneously learning the radiometric normalization mapping on the no-change set of two overlapped images.
Unlike the pseudo-invariant feature-based methods that may fail in the scenarios that lack that kind of feature, we aim to discover the robust no-change set based on the scenario itself.

Latent change noise modeling
In order to model the no-change set in a theoretically grounded manner, we introduce the probability framework. We first consider the values of a shared point on the same geographic location in the two images.
The noise is influenced by the factor that whether the radiometric/color of the point changed or not. Let Z = (Z 1 , Z 2 ) be the 2 dimension binary random variable where Z 1 , Z 2 ∈ {0, 1} and Z 1 +Z 2 = 1. Z 1 = 1 implies the value of point didn't change between the two images while Z 2 = 1 implies the value of point changed between the two images. Suppose p( We provide three examples for ε c and f c . • Case 1: ε c follows mixture of 0 mean Gaussian distribution and f c (X c ) = • Case 2: ε c follows mixture of 0 mean Laplace distribution and f c is mono- • Case 3: ε c follows mixture of 0 mean Gaussian distribution and f c is monotone discrete mapping.
We will later show that first and second cases yield the weighted least square relative radiometric normalization regime and weighted histogram matching relative radiometric normalization regime under moderate condition. The third case yield the robust weighted histogram matching relative radiometric normalization. The case 3 shares the same equation with case 1 except for the f c part.

Condition probability distribution assumption
Let 2 ) for case 1 and 3, • Σ c = (β c1 , β c2 ) for case 2, k is the variance of kth Gaussian distribution of band c, β ck is the scale parameter of kth Laplace distribution of band c and C is the total band numbers. we have • Case 1 or 3: , • Case 2: , βc k . The above equation implies that the change points and no-change points will have different noise levels in the band c. Intuitively, the no-change points set will have a low noise level, and change points will have a high noise level and sometimes may be with a heavy tail. The marginal distribution of the ε c as follows, Let the noises be independent amount bands. Given the radiometric normal-

Marginal log-likelihood
It yields the conditional distribution of Y 1 , · · · , Y C |X 1 , · · · , X C as follows, Consider the situation for all the points in the overlap area with index (i, j) ∈ Then we can formulate a maximum log-likelihood model to obtain the RRN mapping F , parameters Σ and Π as the following,

EM algorithm
EM algorithm maximizes the likelihood objective by uplifting a surrogate objective induced by the latent random variables, EM algorithm uses alternative search of q and Π, Σ, F to uplift E(q, Π, Σ, F ) to maximize the log likelihood.
The expectation step corresponds to the search of q and to obtain new objective: The maximization step corresponds to the search of Π, Σ, F : By assigning Π old , Σ old , F old = Π new , Σ new , F new and repeating the EM steps, the parameters will converge to a critic point of the original log likelihood objective under moderate conditions (Wu (1983)).
In the expectation step, we first compute the following: The γ kij is the posterior distribution of latent random variable Z kij . γ 1ij indicates the posterior probability of the point (i, j) being a no change point. Next Then with the help of alternative search for Π, Σ and F , we can obtain the closed form solution for Π, Σ: As for solving F , case 1 yields the weighted least square regime while case 2 yields the weighted histogram matching regime under moderate conditions.

Case 1: weighted least square relative radiometric normalization regime
The component in equation 40 related to F can be written as follows: and diag(G c ) be the square matrix with diagonal elements equaling G c . Since the objective in equation 48 can be divided into C independent parts in regard to where A T c is the transpose of A c and −1 represents the inverse (or pseudo inverse). Here we obtain the mapping functions F . Theoretically, the alternative search optimization step in the maximization step needs to be repeated several times until Π, Σ and F converge. However, empirically, it works fine if we compute Π, Σ and F once and directly move to the expectation part in the EM procedure. In order to speed up the optimization for Π, Σ, F , we use the stochastic EM procedure (i.e., only use a random subset Ω sub ⊆ Ω to implement each EM procedure) ,and we use the whole points to implement EM algorithm in the final turn.
The objective in equation 52 can be divided into C independent parts in regard to c. We only analyze one part. In order to give an approximation to the solution of f c in this part, we first suppose the whole data satisfy the following assumption, and later we will discuss the situation that disobeys the assumption.
Equivalence of histogram matching and L1-norm image regression.
This assumption claims that the order of the band value should keep in the shared points in both source and target images.
If assumption 2.1 holds, since f c is monotone mapping, x isjsc ≤ x itjtc also implies f c (x isjsc ) ≤ f c (x itjtc ). Then we could rearrange the index of points such Similarly, we have histogram y c = (n y 0 c , · · · , n y T c ) where n y t c = (i,j)∈Ωt y c g ijc . Then the equivalence of match distance (Werman et al. (1985)) 1 of histograms and the objective is demonstrated as following:  Suppose that CP is the cumulative probability function of the histogram.
This is the objective of histogram matching problem, the solution for f c is the following: .
Since discrete monotone f c that minimizes Although not all points might satisfy the color sort assumption, the no-change set intuitively should satisfy this assumption 2 . As for changed points, empirically they will be assigned higher probability to have higher noise level that the γ 0 << γ 1 and β 0 << β 1 . Their weights γ0 β0 + γ1 β1 << 0.5 β0 + 0.5 β1 ≤ 1 β0 is small and would not significantly influence the objective such that the objective is still similar to the objective of histogram matching. Therefore it's still reasonable to use histogram matching to solve F . Theoretically, the alternative search optimization step in maximization step needs to be repeated several time until Π, Σ and F converge. However, empirically, it works fine if we just compute Π, Σ and F once and directly move to expectation part in the EM procedure. In order to speed up the optimization for Π, Σ, F , we use the stochastic EM procedure (i.e. only use a random subset Ω sub ⊆ Ω to implement each EM procedure) and in the final turn we use the whole points to implement EM algorithm.

Case 3: weighted least square histogram matching relative radiometric normalization regime
The component in equation 40 related to F can be written as follows: where f c is the monotone discrete mapping. The above objective is not easy to solve. Let g ijc = Similarly, we have histogram y c = (n y 0 c , · · · , n y T c ) where n y t c = (i,j)∈Ωt y c g ijc . Suppose that CP is the cumulative probability function of the histogram.
as the "surrogate" solution for f c .
Similar to the above two cases, we give the algorithm as the following:

Experiments and results
We designed three kinds of experiments.
The first one is the relative radiometric normalization on complex cases such as cloudy and foggy images. Cloudy and foggy images are the most disturbing images when implementing relative radiometric normalization methods. This experiment demonstrates the auto robust characteristic of our method. Moreover, our method is evaluable regarding the log-likelihood or the variance/scale factor of the noise of the no-change set pixels.
The second experiment demonstrates that our method's relative radiometric normalization mapping can support vegetation change detection and water change detection.
The third experiment illustrates that the no-change set derived by our method can support the building change detection task by reducing the pseudo-change

Materials
We implement our methods on one cloudy image and one non-cloudy image.
The images are obtained by GF 1C satellite, located in the Shandong province of China. We performed the preprocessing steps of RPC orthorectification and geometric correction of these images. These images are multi-spectral, and they are shown in red, green, and blue. They are clipped to the range shown as the source and target images in the figure 1. We exchange the order of these two images to be the target and source image once each. We compare five different methods with its relative radiometric image and its derived no-change set.
We implement our methods on one foggy image and one non-foggy image.
The source image is obtained by GF 6 satellite, located in Henan province of China. We performed the preprocessing steps of RPC orthorectification and geometric correction of this image and down-sample to around 32-meter resolution.
The source image is multi-spectral, and we only select the normalized(ranged from 0-255) red, green and blue bands to implement RRN. It is shown in the figure 6. The target image is an artificial template image of normalized red, green, and blue color. We implement the relative radiometric normalization on the foggy image. We compare five different methods with its relative radiometric image and its derived no-change set.
We implement the HM-RRN-MoG method on additional reflectance images.
The source image F and target image E are obtained by GF 1D satellite, located in China's Shandong province. F was acquired in April 2020, and E was acquired in March 2020. We performed the preprocessing steps of RPC orthorectification, geometric correction, radiometric calibration, and atmospheric correction of these images and extracted the overlapped area. These images are multi-spectral. As for change detection, we compared reflectance/digital num-

Foggy cases
The qualitative results are shown in figure 6     and its potent for land cover change detection task regarding its ability to reducing the radiometric inconsistencies.
According to the figure 7, from the BVW rendered images, we can visually tell that the HM-RRN-MoG method reduces the NDVI and NDWI inconsistencies induced by the radiometric inconsistencies. From the blue-NDVI-NDWI (BVW) rendered change set, we can tell the water(NDWI) changes through the blue-green and blue-purple pixels, the vegetation(NDVI) changes through the bright green pixels, and the man-made urban through orange, khaki and dark green pixels.    is the blue-NDVI-NDWI rendered image using the blue reflectance/digital number as the red band, the NDVI index linearly mapped to 0-1 as the green band, the NDWI index linearly mapped to 0-1. The BVW change set is the remained BVW pixels of the change set. The bright green pixels are with high NDVI and bright blue pixels are with high NDWI. Direct difference method computes the symmetrically difference of the predictions of target image and source image(i.e. T pred buliding S pred buliding ). Difference excluding no-change set method compute the set difference of the direct results and no-change set(i.e. T pred buliding S pred buliding − U no change set ). Difference voting with no-change set method computes the ratio of no change points in each connected component and deletes connected components with a ratio bigger than 80% and is achieved by the region grow algorithm. The set difference with voted no-change set method refines the no-change set by keeping each point in which there are more than 70% no-change points in the (31,31) window centered.

Change detection with no-change set
According to table 5, we can find that the direct difference method possessed the highest recall. Due to the no-change set is of radiometric level rather than object level, some buildings changed to the hardened surface without significant change in the radiometric. When utilizing these kinds of no-change points, the truth changes are negatively influenced. For instance, the set difference excluding no-change set method significantly drops the recall although it has the best accuracy and precision (However, most of no change points indeed indicate the invariance of the object with proper integration.). We find the set difference with voted no-change set best integrate the no-change set, and this method achieves the highest F1 score. The visualization of no-change set supporting the building change detection task is shown in figure 8

Conclusion
In this paper, we present an auto robust relative radiometric normalization method via latent change noise modeling. Our model is theoretically grounded regarding the probabilistic theory and mathematics deduction. Our model possesses the ability to robustly against clouds/fogs/changes. It is evaluable with help of the log-likelihood and no-change set root-mean-square-error as well as relatively accurate. It can effectively reduce the reflectance/digital number, NDVI and NDWI inconsistencies between the source and target image on the no-change set, and its change set may be good guidance for vegetation and water change detection. As for the no-change set generated by our method, we find it can further boost the building change detection tasks when using the proper fusion approach.
Currently, we find that our method may be influenced by the geometric precision of the source and target image. In the future, on the one hand, we would study how to jointly achieve geometric correction and relative radiometric normalization no-change set derivation. On the other hand, we would try to unsupervisedly learn the representation of the oracle target image and implement absolute radiometric normalization through a deep generative model to eliminate the dependence of geometric correction.