{'fraction_non_alphanumeric': 0.02848487232954322, 'fraction_numerical': 0.009442499114765707, 'mean_word_length': 4.853984339014279, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Physics-based correction strategy for large- scale positional deviation of the LED array in Fourier ptychographic microscopy

Fourier ptychographic microscopy (FPM) is a recently proposed microscopic superresolution technique, which combines the advantages of synthetic aperture and phase retrieval to surpass the resolution limited by the numerical aperture (NA) of microscopic objective lens and realize high space-bandwidth product (SBP) imaging. In FPM framework, the position of each sub-spectrum needs to be known accurately to ensure the success of phase retrieval and complex function stitching. Different from the conventional methods utilizing precision mechanical adjustment or data-driven optimization strategies, in this paper, we propose an efficient physics-model based correction strategy for large-scale positional deviation of LED array. Based on the constraint that each LED element is arranged in a regular array, and the relationship between the lateral offset of the defocus specimen image and the illumination direction, the positional parameters of the LED array can be calculated precisely with the aid of subpixel image registration and non-linear regression algorithms. The feasibility and effectiveness of our method are verified through simulations and experiments. The comparison results prove that our method can achieve high quality reconstructed complex image and pupil function even when the LED array is remarkable misaligned, and therefore it can evidently reduce the requirement of the LED array fine adjustment and complex data-driven optimization algorithms in the construction and calibration of FPM systems.


Introduction
Fourier ptychography (FP) [1][2][3] is a recently developed computational imaging technique, which addresses the inherent trade-off between large field-of-view (FOV) and high spatial resolution in conventional optical systems. Based on the corresponding relationship between the illumination direction and the spatial spectrum position, FP can obtain an expanded range of spectrum by providing varying illumination angles, and achieve quantitative phase imaging with a gigapixels spatial bandwidth product (SBP) [4] by utilizing phase retrieval [5] and synthetic aperture [6] algorithms.
Since its first demonstration in 2013, FP has evolved from a microscopic imaging tool of specific architecture to a versatile technique system, embodied in the expansion of implementation methods and application fields. Through modifying or replacing different parts in the conventional FP scheme, different types of implementation cases have been proposed and constructed, such as reflective FP [7,8], aperture-scanning FP [9], X-ray FP [10], multicamera FP [11], et.al. Meanwhile, thanks to its large SBP and phase imaging capabilities, FP has been applied and verified in many application fields, including quantitative 3D imaging [12][13][14], digital pathology [15,16], aberration metrology [17] and fluorescence imaging [18], et.al. In a typical FPM system, an LED array is always used to provide angularly varying illumination. During the data acquisition process, LEDs are turned on sequentially to provide different angular illuminations and a camera is used to record the corresponding low resolution (LR) images whose spatial frequency ranges are determined by the NA of the objective lens and the illumination wavelength and wavevectors. In the original FPM model, we suppose each LED element is a point source that emits quasi-monochromatic light, and the whole LED board should be aligned. The former can be achieved by choosing LED element with a narrower spectral bandwidth and a smaller light-emitting area. Nevertheless, the positional deviation of the LED board is unavoidable during the process of constructing or modifying FPM systems, and the positional deviation will degrade the quality of the reconstruction by making the sub-spectrum position used in the phase retrieval algorithm inconsistent with the wavevector corresponding to the actual illumination directions. One conventional way to deal with this problem is to pre-calibrate the LED array by mechanical adjustment devices [19,20], but aligning the LED array accurately is a time-consuming task and needs precise and multidegree of freedom mechanical adjustment devices, which are always bulky and expensive. Many data-driven optimization approaches taking use of the LR data redundancy are also useful solutions, such as pcFPM method [21], two-part calibration method [22], QG-FPM [23], and tcFPM [24], et.al. However, all these methods use the intensity distribution characteristics of the LR images to seek an optimal positional parameters estimation of the LED board, whereas besides the positional deviation, the LR images' intensity distributions are also affected by illumination intensity fluctuation, objective aberrations, random noise and other kind of systematic errors. The ways in which all errors affect the LR images are mixed together, result in extracting positional deviation from the LR images constraint become an illposed problem. It is thus difficult for these methods to be performed with the optimization and correction methods of other non-ideal parameters. In addition, the data-driven optimization approaches also have the problem of time-consuming and easy to fall into local traps.
To eliminate the influence of other non-ideal systematic parameters and accurately obtain the positional deviation of the LED board, we propose a correction method with an explicit imaging physical model, called pbcFPM. Based on the constraint that each LED element is arranged in a regular array, and the relationship between the lateral offset of the defocus specimen image and the illumination direction [25][26][27], the positional parameters of the LED board can be calculated precisely with the aid of subpixel image registration and non-linear regression algorithms. Simulations and experiments have been conducted to demonstrated the feasibility and effectiveness of pbcFPM for large scale positional deviations. The proposed method can significantly improve the quality of the reconstructed complex amplitude images, obtain the objective pupil function without influence of positional deviations, and therefore evidently reduce the accuracy requirements of the LED array board in the process of building FPM platforms.
The remainder of this paper is organized in the following manner. In Section 2.1, we analyze the effect of LED positional deviation through the reconstructed results, in Section 2.2, we establish a mathematical model between LED positional parameters and LR image offset, and in Section 2.3, we give the flow process of the method. The effectiveness of pbcFPM is confirmed with simulations and experiments in Sections 3 and 4, respectively, and the summary and further discussion are given in Section 5.

Positional deviation in FPM system
A typical FPM system generally consists of an LED array, a low-NA objective lens, and a monochromatic camera. When acquiring the raw data set, the LEDs are turned on in a sequential manner to provide angular-varying illuminations. When LEDm,n (m, n are the indexes of rows and columns) is working, an plane light with a wave vector of ,, m n m n uv is generated to illuminate for a small segment of the specimen , and the camera is used to acquire the corresponding LR intensity image , ( ', ') c mn I x y . Subsequently, these raw images are processed by the conventional FP reconstruction algorithm to obtain the HR complex amplitude information of the specimen. The reconstruction process is divided into five steps.
The first step is to initialize of the estimation of specimen complex amplitude 0 ( , ) o x y and the pupil function 0 ( , ) P u v , then the spectrum estimation of specimen is 00 ( , ) , where ' ' denotes the Fourier transform. Secondly, using the current estimation of pupil function and spectrum of specimen to obtain the LR image estimation y is formed. It is worth noting that the accurate position of each sub-spectrum needs to be known when updating the HR spectrum in the fourth step for high recovery quality. In other words, the illumination wave vectors determined by the position of each LED element and the distance between LED board and specimen need to be precisely known, which put high requirements on reconstructing and modifying FPM systems. Fortunately, some improved recovery strategies have been proposed to relax the demands. EPRY algorithm [28] is a helpful method to get better solution by updating HR spectrum and pupil function simultaneously. When the deviation between ideal and actual positions of sub-spectrum is not very large, it can be corrected by EPRY algorithm. However, as the deviation becomes greater, the performance of EPRY algorithm will decrease, and resulting in wrinkles artifacts in the reconstructed amplitude and phase, the quality of the result is seriously degraded.
Here, we demonstrate the influences of positional deviation of LED array by simulation. For directly analyzing the relationship between recovery quality and positional deviation, we assume the LED array is only moved along the x-axis, which is a pre-defined direction in the transverse plane, and use Δx to represent the degree of deviation. We use EPRY algorithm to recovery the intensity and phase when Δx varies from 0 to 2000μm, and the results is shown in Figure 1. Figure. 1(a1) and (b1) shows the intensity and phase ground truths of the specimen. When Δx is equal to 200μm, the reconstructed intensity and phase are shown in Figure. 1(a2) and (b2) respectively. Although the images are blurred and the center of the phase becomes black, the change is not disastrous, and distribution characters of both the intensity and phase are still distinguishable. When Δx increases to 800μm, the reconstructed intensity and phase are shown in Fig. 1(a3) and (b3), respectively. Although most of the intensity information can still be distinguished, wrinkle artifacts appear in the intensity and phase images and the phase information has been seriously distorted. When Δx reaches 2000μm which has resulted in the wrong choice of center LED, obviously black artifacts and some unexpected phase information have arisen in intensity image shown in Fig. 1(a4) to degrade the imaging quality, and the phase information shown in Fig. 1(b4) is more terrible because whole image is completely submerged and indistinguishable.
By intuitively observing the HR reconstructed results with the same Δx, it can be found that the reconstructed phase quality degradation is more serious compared to the reconstructed intensity. That is because the sub-spectrum error in the Fourier domain is directly related to the phase information in the spatial domain. Figure.1 (c1) and (c2) gives the relationship between the root mean square error (RMSE) of the reconstructed and real intensity and phase and the positional deviation Δx. The RMSE of the phase is obviously larger than intensity. In addition, it is noteworthy that the RMSE is not linearly related to Δx, but remains stable when Δx changes in a certain interval (e.g., the RMSE of intensity is about 0.2 when Δx varies from 1400μm to 1800μm), and the RMSE increases sharply when Δx exceeds a certain threshold (e.g., 800μm). The abnormal phenomenon is caused by the discretization of the Fourier domain coordinates when reconstructing the HR image. If the deviation between the real and ideal positions of sub-spectrum is no more than one pixel in Fourier domain, the reconstruction quality of the images is stable and the RMSE will around a constant. However, if the deviation is more than one pixel, then the recovery quality will start to drop sharply. Moreover, in an actual FPM system, the LED array will also have other positional deviations besides the x-axis shift, such as the y-axis shift and the rotation along optical axis, et.al. The recovery quality will degrade more seriously under the combined effect of these deviations, so it is very significant to correct the positional deviation of the LED array.

Image Offset Model
A typical FPM platform and the corresponding LED array with positional deviation are shown in Fig. 2. The blue LED element in Fig. 2(c) is the considered center LED but point O is the actual center with respect to the specimen, and the orientation of LED array (the black dashed line) deviates from the x-axis. In fact, the high precision of LED board manufactured by existing technique enables us to consider the arrangement of LED elements regular, and FPM system is always placed on a horizontal stable. We can assume the LED array shown in Figure  2  When turning on m,n LED to provide illumination, the illumination light can be considered as a plane wave for each small segment of the specimen, and the wave-vector ,, where  denotes the wavelength of the illumination,

( )
, cc xy is the center coordinate of each small segment, and h is the distance between the specimen and the LED array.
When the specimen is in the focus position, the LR image acquired by the camera can be expressed as where ( ', ') xy is the Cartesian coordinate of the camera plane, and ( , ) P u v is the coherent transfer function of the FPM system.
When the specimen is in the defocus position, the LR image can be described as where A is the magnification of the objective lens, and , mn w denotes the wave vector along the z direction and can be calculate as The first terms in ( , , ) d H u v z are phase change in Fourier domain and will result in offset in space domain, and the third term will make the quality of the LR image degrade. We take no account of the effect caused by the third term and derive the relationship between the defocus and focus images as v ) for a certain small segment of specimen. Therefore, the illumination direction of each LED element can be obtained by calculating the image offset.

Correction strategy
The process of pbcFPM is shown in Figure 3. In the beginning, we adjust the specimen to the focus position with the focusing knob as shown in Fig. 2(a), then turn on LED0,0 and the caprure the corresponding LR image 0,0 ( ', ') c I x y . In FPM system, determining whether the specimen is in the focus position or not is an easy task; when the specimen is in the focus position, the LR image acquired will not shift regardless of the angle of the incident light because d z is zero in equation (6). Therefore, if the LR image does not shift with different LED illumination, then we can render the specimen is in the focus position. In this step, although it is feasible to choose to any LED element in the bright field as the light source, considering that the light emitted from LED0,0 has better symmetry, and the captured LR image's energy also has better symmetry, so the LR image 0,0 ( ', ') c I x y is better to be the reference image.
Secondly, we adjust the specimen to the defocus position that is 200μm away from the focus position, then turn on the LED elements within the bright field (BF) of the objective lens to provide illuminations and capture the LR images ,, ( ', '), , 2,1,0,1, 2 c m n d I x y m n =− . In this step, the defocus distance is a crucial parameter. The defocus distance can't be too small, otherwise, the calculated image offset will have a low signal-to-noise ratio because the offset of the image is too small. In contrary, the image will be seriously distorted because of the defocus aberration, and the calculated offset is not accurate. In this paper, we take 200 d zm  = , where the image offset is enough and the image quality is also high. Furthermore, pbcFPM is based on the accurate calculation of relative image offset, and the geometrical characters of BF images are much better than DF images, so we choose the corresponding LED elements to providing illuminations. xy  between the defocus images and the reference image. Considering calculating the offset of the images within the whole FOV is unnecessary and time-consuming, we choose to calculate the offset of a small segment, which can be a stripe on the resolution target, a small hole, and a cell, et al. Furthermore, because the grayscale of background will exert an effect on precision of calculating the image offset, we binarize the selected small segment and then calculate the offset of the center of mass with subpixel image registration algorithm.
After getting the image offsets, we utilize non-linear regression algorithm to obtain the positional parameters of the LED array. Mathematically, the non-linear regression process can be described as 2 22 , , , m n m n uv in the process of updating HR spectrum. However, when the LED array has a considerable deviation, the original optimal center LED may be replaced by other LEDs around it, and then the convergence of the solution will be degraded if the recovery process is iterated according to the original spiral order. In the process of reconstructing HR image with pbcFPM, we first calculate the distances between the corrected positions ,, ( , ) m n m n uv and zero frequency's position, and then rearrange the update order from nearest to furthest according to the distances to ensure fast convergence even under large positional deviations.

Simulation
Before applying pbcFPM strategy to correct the positional deviation of actual FPM platform, we first validate its effectiveness with simulations. The parameters in the simulations are chosen based on the actual system. We utilize a 21×21 programable LED array as the light source to provide angle-varied illuminations. The distance between adjacent LED elements is 2.5mm, and the wavelength of the illumination light is 470nm. The NA of the objective lens is 0.1, and the magnification is 4. The pixel size of camera is 2.4 .
As mentioned in section 2.3, we use the offset of a small segment to represent the offset of image. Considering that there are many rectangular stripes on resolution target, we employ a square hole with a side length of 50 as the specimen and place it beneath the LED array and the distance is 92mm, and the defocus distance d z is 200 . When simulating the LR images, the positional deviations are introduced artificially and taken in the range of ( 2.5 ,2.5 ) In an actual FPM system, the positional deviations can be adjusted to the range by rough adjustment with mechanical adjustment devices. In addition, if the FPM system does not equip with any mechanical adjustment device for the LED array, we can choose the optimal LED element as the center one in which the transversal deviation is smaller than the deliberately set maximum value according to the characters of the spots acquired by the camera; and even unskilled users choose the wrong LED element as the center one, we can also rearrange the update order when reconstructing the HR images to remain fast convergence and good recovery quality.   Figure. 4(a) shows the small square specimen that is used to calculate the image offset. Figure. 4(b) is the LR image when the specimen is in the focus position and illuminated with the center LED element. Figure. 4(c) shows the image when the specimen is in the defocus position and illuminated with the center LED element. Figure. 4(d) shows the image when the specimen is in the defocus position and illuminated with the LED element of row 2, column 2. Figure. 4 To verify the performance of the pbcFPM under other large positional deviations, we continue to perform the simulation with the positional parameters within the setting range. Table 1 shows the simulation results when the LED array has different representative deviation parameters. The parameters obtained by pbcFPM are still in good agreement with the actual parameters.
Next, we utilize the obtained positional parameters to correct the position of the subspectrum position in the Fourier domain during the process of reconstructing the HR image.

Experiment
To evaluate the effectiveness of pbcFPM experimentally, we used a USA-1951 resolution target as specimen to compare the reconstructed intensity distribution and pupil function with conventional FPM and pbcFPM respectively.
We conduct the experiments with the FPM system shown in Fig. 2(a). The light source is a 21×21 LED array, the distance between adjacent LEDs is 2.5mm, the illumination wavelength is 470nm and its bandwidth is 20nm, the NA of the objective lens is 0.1, the magnification of the objective is 4. We utilize a camera (FLIR, BFS-U3-200S6M-C, sensor size 1", dynamic range 71.89 dB, pixel size 2.4μm) to record the LR images. Figure 6 shows the performance of EPRY-FPM and pbcFPM with different positional deviations. The full FOV LR image is shown in Fig. 6(a). We use the center-of-mass offset of a small rectangular shown in Fig. 6(b1) to represent the offset of the image for simplifying calculating and accelerating the speed of subpixel image registration algorithm. Fig. 6(b2) is a small segment of Fig. 6(a) that is used to reconstruct the HR images. Fig. 6(c1) and (c2) show the reconstructed HR intensity images and pupil function without positional deviations. The HR intensity image owns excellent imaging quality and the reconstructed pupil has good symmetry. Next, we utilize the mechanical adjustment device to translate the LED array by 2 mm along the y-axis, and the reconstructed intensity image and pupil function with EPRY-FPM are shown in Fig. 6(d1) and (d2), respectively, where wrinkles artifacts appear in the reconstructed intensity image, but the degradation of imaging quality is not very obvious, which is because EPRY-FPM will identify the positional deviations of the LED array and consider it as a kind of aberration. Therefore, EPRY-FPM couples the positional deviations of the LED array into the reconstructed pupil function, then the pupil in Figure. 6(d2) is distorted, so the pupil obtained is not reliable. In addition, the correction ability of EPRY-FPM is also limited, when the positional deviation is remarkable large, the effectiveness of EPRY-FPM will be lost. Similarly, we use the mechanical adjustment device to move the LED array along the x and y axis respectively by 2mm, at this time the reconstructed HR intensity image and pupil function with EPRY-FPM are shown in Figure 6(f1) and (f2). The quality of the intensity images severely degraded because of the obvious wrinkle artifacts, and the details is difficult to distinguish.  In addition, to verify the generalizability of our method, we use paramecium slice as the specimen, then randomly move and rotate the LED array to an arbitrary. position. We compare the reconstructed intensity, phase, and pupil function before and after utilizing pbcFPM. Fig.  7(a) shows the LR image of the full FOV acquired by the camera, and Fig. 7(b) is the small segment used for reconstruction. The reconstructed intensity, phase, and pupil function with EPRY-FPM are shown in Fig. 7(c1)-(c3), respectively. Many wrinkle artifacts appear in the reconstructed intensity information, and the reconstructed quality of the image is severely degraded, especially the reconstructed phase image's contrast is severely down and the phase information is severely lost. Furthermore, the reconstructed pupil is also severely distorted. Subsequently, the positional parameters of the LED array are calculated with pbcFPM, and the parameters obtained are 1.680 x mm = , 0.089 y mm = The reconstructed intensity, phase, and pupil function are obtained after using these parameters to correct, as shown in Fig. 7(d1)-(d3), respectively, the artifacts caused by the positional deviations vanish in both the reconstructed intensity and phase images, and the imaging quality of the image is highly improved.

Discussion
In this paper, we propose a positional correction method named pbcFPM that yields highquality reconstructed intensity, phase, and pupil function, and the effectiveness of pbcFPM is verified by both simulations and experiments. Different from the existing complex data-driven optimization strategies, pbcFPM has an efficient physical model that can separate the positional deviations from many coupled systematic errors. We utilize the relationship between the offset of the defocus image and illumination direction to firstly obtain four parameters to describe the position of each LED element, and then precisely calibrate the position of subspectrum in the process of stitching LR measurements in the Fourier domain. The compared recovery results with large scale positional deviations prove the excellent performance of pbcFPM. In addition, if the positional deviations are remarkable large that can cause wrong choice of the center LED, our method can still achieve a good recovery results by rearranging the updating order. Thus, our method can handle arbitrary positional deviations.
Although pbcFPM can correct large scale positional parameters and reduce the system's accuracy requirements, the image offset is in the order of tens of microns, then the accuracy of the focusing knob is highly required in the process of adjusting the specimen to defocus position. It is necessary to ensure that the image will not be or just slightly shifted during the focusing process. Of course, for the focusing device with poor accuracy, it can also be feasible because the image offset caused by the mechanical adjustment can be calibrated before the experiment.