{'fraction_non_alphanumeric': 0.024050836550836553, 'fraction_numerical': 0.006113256113256113, 'mean_word_length': 4.861857614332862, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Investigations on infrared-channel-image quality improvements for multispectral imaging

The improvement of image quality in the infrared range on silicon-based sensors is one major topics using these long wavelength channels for geometric measurements. The reason behind the bad quality of infrared images in comparison with the visible sampling range is explained by the wavelength response dependency of the silicon. Photons are able to either to pass the sensitive range of the pixel or can tunnel to the neighbor pixel. This effect leads to blurred images, which will not only increase the uncertainty of measurement, but also the aesthetical quality of the image. In this paper, methods to improve the image quality using blind convolution as well as a special infrared focusing to improve the sharpness will be presented.


Introduction
Multispectral and hyperspectral imaging technologies allow detecting several features in industrial inspections tasks using the different spectral channels [1] its applications range from cultural heritage, arts, remote sensing, environmental monitoring, medicine, biology, food quality control, military applications, factory automation and manufacturing [2]- [7]. Besides an accurate registration of images acquired at different wavelengths for the spatial-spectral utilization of multispectral data [8], the consideration of the semiconductor characteristics is a big issue. Therefore, in [9] a method for the characterization of pixel inhomogeneity's caused by the manufacturing process and read out process is proposed. These investigations show how the specific data can be processed to compensate the inhomogeneity's as function of the current integration time. A fact which is actually not discussed in these works, is the crosstalk characteristics between the pixels in dependence of the wavelength. The material characteristic plays an important role. The characteristics of penetration depth versus absorption of photons for different materials are given in [10]. It is shown in [10] and [11] that in the near infrared region (NIR) the generation of electrons will happen in an range of tens to hundreds of microns inside the semiconductor. Therewith it is possible that the electrons flow into neighbor pixels, which is presented in [12]. Finally, these effects in the modular transfer function of the image sensor and the images have a blurred appearance as well as blurred information across edges, which account for the deterioration of the accuracy of geometrical measurements in the NIR regime: blurred images have less sharp edges (see Figure 2) and edge detection based algorithms rely on precise estimation of edge's position. Unfortunately, this effect has a statistical behavior so a discrete model based deconvolution of the blurred image is not possible. In this paper, an approach to improve this characteristic using a statistical method based on a combined image restauration algorithm is part of the discussion. Other sources of instabilities and how to tackle them are discussed in [1], [8]- [9], [13].

Analysis of the blurred images in the infrared region
The blurred images caused by the electron generation of the incoming photons and the flow inside the neighbour pixel regions inside the deep regions of the semiconductor are hardly measurable without extensive setups. Nevertheless, for practical experiments the estimation of the sharpness on real pictures can be evaluated. For the investigations in the infrared regions a multispectral filter-wheel camera was used to generate the image data in the VIS-NIR region from 650-950nm. To minimize the defocus caused by the optics in the different spectral channels, a refocus for every wavelength has to be made inside the multispectral imager [1]. Therefore, a high precision sensor actuation inside the multispectral camera is used [13].

Autofocus calculation
As mentioned in the introduction, the multispectral imager offers the possibility to correct the longitudinal chromatic aberration using a linear moving stage to set the image sensor in the correct position. To ensure a certain repeatability and to ensure comparability between the different infrared channels, Laplace approach for the calculation of the best focus position was applied. The Laplace approach uses the Laplace energy as given in equation 1: . (1) A linear moving stage inside were actuated and the M Lap were calculated for each position. The optimal focus position is the sensor position with the highest Laplace energy (Figure 1 c). After the focus procedure the Sensor was driven to the optimal position. The different focus positions along the optical axis can be stored inside the camera [1]. The depicted images ( Figure 1) show as an example the difference between an NIR-image and a VIS image. Next step is to find a method to get the blured information in an evaluable value.

Approach for the evaluation of the image sharpness
After the images were refocused for each wavelength, the evaluation of sharpness was processed using the edge transition of a chrome glass target depicted in Figure 1. Therefore, search lines along the radii will be fitted on the outer contour of the circles. The result of this method delivers a grey value transition for each line (normalized grey value (blue) Figure 2), which must be processed.
Using a normalization method, the edge transition was converted to a normalized region between zero and one. With these values a best fit using a gaussian fitting method was applied to get a continuous edge transition description. Hence a Heaviside function (Figure 2 black) will be fitted over the real edge transition (Figure 2 fitted gaussian error function (red)) and the grey values were normalized to that function (min-max grey value). After that, an iterative algorithm tries to get the same area on the left and right side of the Heaviside function shifting itself along a virtual x-axis. If the areas are equal, the width is calculated by the pixel amounts, which are involved getting the areas equal (this is also   Characteristic edge width behaviour starting in VIS up to the NIR wavelength range, where edge width becomes large due to cross-talk between pixels and the transparency of the semicobdctor for higher wavelengths. The seaminly blured image found at 400nm can be explained by two factors: low photon count on the UV region (light source depentent), which makes necessary higher integration times and with that, higher noise content; and the fact that the Si sensor is less sensitive to this region of the spectrum [1]. Figure 6 shows the values representing the edge width in the NIR channels. As a comparison, the width in the visible wavelength region is less than 2.5 pixels. To improve the system behaviour in the NIR range, discussed in section two, a deconvolution method was chosen to improve performance in the NIR. First of all, a method to estimate the point spread function is necessary. To overcome this problem two methods were tested. The direct PSF-estimation, which is based on the calculation out of a white noise pattern, does not perform accurate. The reasons behind that can be studied in the quality of the white noise target.

Point spread function estimation using blind deconvolution
The second approach uses a blind deconvolution models using the expectation-maximization optimization (EM) [14] and the maximumapostori estimation (MAP) [15]. While the MAP method calculating the PSF and the corrected image simultaneously, the EM method calculates the PSF and the image correction separately. This leads to lower number of parameters, which must be calculated during for the EM PSF estimation. Furthermore, the image restauration algorithm can be chosen independently from the PSF estimation method. After a few investigations in the 900nm infrared channel the EM method delivers more reasonable PSF-functions. The generated PSF using MAP delivers approximately the Dirac function as result. With this behaviour, a performant image restauration cannot be achieved. Furthermore, the lower number of parameters of the EM method leads to a better function performance which leads to the final decision for the EM-processing method.

Spatial PSF Processing using EM method
The PSF has a spatial dependency. This leads to the necessity of calculating the PSF per the described method within a defined image-block region [16]. With the spatial information of the local PSF distribution inside the sub image blocks, the image is convoluted using the MAP optimization. Taking into consideration the spatial variation of the PSF, the acquired calibration image divided into several sub-image blocks. By means of the blind deconvolution, a PSF is calculated for each subimage block (as shown in Figure 4). Under the assumption that the PSF is piecewise continuous and a soft function, that means, it varies slowly along the x and y coordinates, an algorithm for the stitching and merging of neighbouring PSFs is applied. Figure 4 illustrates how the algorithm works: The pixels of the estimated PSFn for different picture blocks are transformed in new images in the following manner. The pixels at the same position within each PSF are arranged to a new block, as shown in Figure 4, where each pixel from the right handed image comes from the calculated PSF for the most upper-left pixel. Each one of these blocks is then filtered with a low-pass filter (Equation 2 shows the core of the filter). This filter reduces the stochastic error between neighbouring blocks. ). ( Filtering ensures a certain uniformity between adjacent PSFs. After this filtering, the pixels are reset to their original positions in the PSFn. Subsequently, a threshold-based filtering of the PSF is performed by setting the values which lie below a specific location-dependent threshold value to zero. Thereby, the noise which is close to the limit of a PSF and and must have a low value is reduced. The thresholding windows are defined as follows: where N is the size of the window. This Turkey-window is chosen because the width of the top of the window can be chosen by setting the parameter α. The threshold operation is then performed by: where ̃ is the maximum value of the unfiltered PSF ̃( , )of each block, and ( , ) are the filtered values. Finally, the PSFn are normalized so that the sum of all PSF values is equal to 1.

Image restauration using spatial PSF estimation
Regarding the location-dependent PSF estimation, each image is also split into several image blocks and each image block is restored with the corresponding PSF. There are already a variety of methods for image restoration, for example, the Richardson-Lucy method and the Wiener filtering. These methods can allow a certain increase in the sharpness of the image, but cause annular artifacts in the vicinity of the edges. In order to suppress these effects, the method used in [17] is used in this work, which is based on MAP optimization. For the subsequent investigations, the original code is used by the author.

Experimental results and Discussion
In order to evaluate the improvement of the sharpness of the restored images, edge detection on five search lines was performed both on the original and restored images. The positions of the search beams are indicated in Figure 5 (the search beam for each edge detection is shown in yellowright handed image). Point 1 is at the center of the image, and points 2 to 5 are at the four corners of the calibration target. Figure 6 shows the edges of the original images and the restored images. It shows that the edge width can be significantly reduced by image restoration. The average edge width is reduced from 4.15 pixels to 2.49 pixels, which means that an increased image quality, i.e. sharpness, can be detected. The experimental results demonstrate that the image restauration works in a proper way. An improvement of more than approx. two pixel was achieved in the best case and approx. one and a half pixel in the worst case. These results lead to better system performance for measurement tasks, which will be realized with NIR-imaging. Nevertheless, the error on the target depends additionally on the optics that will be used for the application.

Acknowledgements
We thank the federal ministry of education and research for the strong support of this work. The work is related to the project Qualimess next generation (03IPT709X).