{'fraction_non_alphanumeric': 0.03058766859344894, 'fraction_numerical': 0.006984585741811176, 'mean_word_length': 4.615280594996619, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
PSF Estimation for Restoration of Zoom-Blurred Endoscope Images

In order to remove zoom blurring caused due to the movement of the endoscope in the depth direction, it is necessary to obtain the point spread function (PSF) from the degraded image. However, the PSF of a zoom-blurred image has not been clearly modeled because zoom blurring depends on the position of the image. In this study, we propose a method to estimate the PSF of a zoom-blurred image and conduct a fundamental experiment to confirm its validity.


Introduction
The demand for endoscopes is increasing for diagnoses of the digestive system. However, it is necessary to obtain clear images by removing image blurring caused by endoscope movement so that medical doctors may correctly diagnose a disease.
In the literature, there are some methods for removing image blurring, i.e., using a gyroscope or multiple images [1]- [4]. However, these methods cannot be applied to the endoscope device as it cannot be equipped with the necessary hardware and cannot obtain multiple images. Although some other methods that correct a blurred image using the relative motion between the camera and the object have been reported [5]- [8], they also cannot be used, since their main assumption is that only spaceinvariant blurring occurs (Fig. 1). In contrast, in our problem, the direction and the intensity of zoom blurring may vary with the position in a given image (Fig. 2).
In order to solve this problem, we have proposed a method that removes zoom blurring by dividing the degraded image into small regions, where zoom blurring is approximated to be uniform [9]. However, the image resolution of the restored image is not sufficiently improved by this method for the images that are actually obtained in experiments. This is because an actual zoomblurred image is affected by zoom blurring and the defocus effect; that is, an imaging object is out of focus when an imaging device moves in the depth direction. Therefore, the previous method could not obtain a clear image since zoom blurring and defocusing are not simultaneously compensated. In this study, we modeled the PSFs to remove both types of blurring. Furthermore, we performed some experiments to confirm the validity of the PSFs generated with our simulation.

Zoom-Blurred Image Generation Method
To generate zoom blurring, original images were gathered at regular intervals and divided by their total number (Fig. 3). The magnification R of each enlarged image can be calculated as in equation (1), where L is the object distance, v is the moving speed of the imaging device, t is the exposure time, n max is the total number of images, and n is the number of enlarged images. In this simulation, the imaging device was assumed to move towards the imaged object and the total number of the images was set to 100. Furthermore, a bilinear interpolation was also used in the image enlarging method.

Defocus Generation Method
The convolution operation of the defocus PSF ) , ( y x d and the image ) , ( y x f created by using the above zoomblurred image generation method is used to account for the defocus effect.
The result ) , ( y x g is the new zoom-blurred image including the defocus effect. To approximately compute ) , ( y x d we used the following two-dimensional Gaussian function: where the standard deviation σ is the defocus parameter and depends on both the defocusing distance and the focal length. Therefore, we need to obtain σ at each defocusing distance and model the defocused PSF (defocus PSF), accordingly. The σ parameter is determined from the recorded images using the following procedure.
First, we obtain an image at the focal position to generate multiple degraded images with different defocus intensities using equations (2) and (3), and different values of σ (virtual degraded images). Next, we acquire images at each defocusing distance. Finally, these captured images are compared with the virtual degraded images (Fig. 4) using the mean squared error (MSE) as an evaluation metric, and the value of σ is set as one where the MSE is minimized.

Defocus parameters selection
Before evaluating the zoom-blurred image generation method, we have to obtain the defocus parameters as a function of the defocusing distance. The specifications of the imaging device used in this experiment are summarized in Table 1. To estimate σ, the images were collected at 15 mm from the focal position every 0.5 mm.

Zoom-blurred image generation
To confirm the validity of the generated PSF with our simulation, we acquired the zoom-blurred images by moving the camera in the depth direction. In this step, in order to adjust the imaging device's moving speed and object distance, we moved the camera by using an actuator and controlled the exposure position by using the camera's trigger function and function generator (Fig. 5). The specifications of the actuator and the conditions of the simulation and experiment are summarized in Tables 2  and 3, respectively. In conditions (i) and (iii), we reproduced the equivalent situation of minimum and maximum zoom blurring of a real endoscope's in-depth movement; in condition (ii), we simulated standard zoom blurring during endoscopy. In this step, we used the original image (Fig. 6) along with white dots, the image positions of which are listed in Table 4.

Modeling the defocus parameters
The values of σ at each defocus distance are shown in Fig. 7; these values did not change within 1.5 mm of the defocus distance since this area is within the depth of the field. When the defocusing distance exceeded 2.0 mm, σ changed proportionally with the defocusing direction. The collected data, shown in Fig. 7, helped us determine the function of σ formalized in equation (4). In order to describe the main parameters of the zoom blurring, the details of which will be discussed later, we need to express the defocusing distance as the product of v and t.

Evaluation of PSF due to zoom blurring
For each condition in Table 3, the values of σ obtained using equation (4) or Fig. 7 are summarized in Table 5 (called the modeled σ). It is possible to estimate the performance of this modeled σ function (equation (4)) by comparing the calculated σ, based on the full width at half maximum (FWHM; expressed in equation (5)), with the dot degradation at the center position (a) of the original image, since the defocus only occurs around the central area.

2
In FWHM   (5) The values of the calculated σ for each condition are summarized in Table 6. The comparison of Tables 5 and 6 shows that the maximum error in σ is 0.05 a.u.. This confirms the validity of the modeled σ function for all conditions.
In order to evaluate the direction and the intensity of the zoom-blurred images generated by using our numerical simulation, we compared the shapes of the PSFs at positions (b)-(d) in the original image. Then, we obtained the luminance profiles along the extension direction of the PSFs at each position for both the simulation and the experiment, which are shown in Figs. [8][9][10]. The analysis of the shapes of all PSFs, confirmed that the extension directions of the PSFs of the simulation approximately coincided with the experiment ones. In addition, the luminance profiles of the generated PSFs for the simulation also coincided with the experiment ones, except for condition (iii) at position (d), as shown in Fig.  10, where the moving distance of the dot is 17 pixels. Since for the other conditions the moving distance of the dot was within 12, we can conclude that our numerical simulation, for the PSF modeling of zoom blurring, was valid when the moving distance of each dot in an image was less than 12 pixels.

Conclusions
In this paper, we proposed a method for generating a zoom-blurred image by numerical simulation. We also tested the PSF generated with this method through multiple experiments. In this study, we found that the luminance profiles obtained through both simulation and experiment almost coincided with each other for condition (ii), and we were able to generate typical zoom blurring present in endoscopy. Therefore, we quantitatively evaluated the validity of our previously proposed imagerestoration methods by using the zoom-blurred image generated with our simulation.