{'fraction_non_alphanumeric': 0.014457734283758206, 'fraction_numerical': 0.007893359107567154, 'mean_word_length': 4.973538609574212, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Using NUFFT in nonuniform sampling Fourier transform spectrometer and the comparison with conventional interpolation FFT method

Resampling by interpolation is the traditional method to process sample in nonunform sampling Fourier transform spectrometer. Nonuniform discrete Fourier transform is an alternative to interpolation that has not been overlooked before. With the aid of experiment, we systematically compare the NUFFT method with resampling by interpolation FFT method in nonuniform sampling Fourier transform spectrometer. We found that NUFFT is comparable to interpolation in spectral profile and spectral noise levels and is better in spectral amplitudes. We also found that It has significant advantage in under-sampling and anti-aliasing property which is offered by the unique non-periodic nature of nonuniform sampling. It is faster and consumes less computer memory in our python implementation. Overall, we found that NUFFT is superior to traditional interpolation method with mostly better performances as well as additional capabilities.


Introduction
Michelson interferometer and its many variants are the foundation of many optical and non-optical spectrometry designs. One of the types is the Fourier transform spectroscope. It uses a moving mirror in one of the arms of Michelson interferometer to generate interference signals and uses a single pixel photodetector to record the interferogram. It is one of the most common types of optical spectrometers and has seen applications up to ultraviolet range at 40nm (Nelson de Oliveira, Nature Photonics, 2011). Although its principle is relatively simple, it has many advantages unrivalled by other techniques. It has high throughput (Jacquinot advantage), multiplex capability (Felgett advantage), intrinsic wavelength calibration and the best instrumental line-shape of any spectrometer (Connes advantages)[ Davis, S.P., et al, Academic Press, 2001.]. It is especially advantageous in infrared region where it is the most widely used technique in infrared spectral systems.
One of the challenges in Fourier transform spectroscopy is the difficulty to obtain evenly spaced sample in optical path difference space due to the difficulty in maintaining a constant mirror moving speed precisely. One of the simplest solutions is to sample non-uniformly instead and then use interpolation to resample the data into uniform ones such as the example here[Yijian Meng, Journal of Modern Optics]. This method has the advantage of having least requirement on hardware and can be readily realized with off shelf components and thus can be desirable in many situations. But in such cases of nonuniform sampling Fourier transform interferometers there also exists another overlooked method to process the data to obtain the spectrum which is to use the nonuniform discrete Fourier transform theory.
Nonuniform discrete Fourier transform is an old concept that has existed for a long time [Marvasti, Farokh, ed. Springer Science & Business Media, 2012.]. It has many applications in a diverse range of fields ranging from imaging processing to numerical solutions of differential and integral equations(Qing Huo Liu, IEEE Transactions on Geoscience and Remote Sensing, 2000.), filter design(S. Bagchi However, while nonuniform Fourier transform has been seen extensively in more complicated interferometric systems like MRI, to our best knowledge its application in optical spectrometer systems has been very rare and has not been used in arguably the simplest type of Michelson interferometer based spectrometers, the single dimensional optical Fourier transform spectrometer with moving mirrors. And its advantages or disadvantages over the traditional interpolation based uniform Fourier transform method has not been well studied. Studying this simplest case of nonuniform Fourier transform not only can help improve the technology of Fourier transform spectroscopy but can also have implications in other applications that employs interferometers. Thus this paper will try to experimentally study the difference and advantage or disadvantage of nonuniform Fourier transform method over the traditional interpolation method in one dimensional optical Fourier transform Michelson spectrometer. There has been one theoretical study on comparison of NUFFT over the interpolation method (David A. Naylor, Proc. SPIE 5546, Imaging Spectrometry X, 2004). It shows that NUFFT method is superior to interpolation method in simulation. But this study is solely based on simulation and not backed up by experimental data. This study is also limited in scope and there remains a lot more to be studied. And its scheme of NUFFT method is also different to this paper too in that it employs iteration which although may improve spectral accuracy would also incur heavy speed penalty. This paper will build a 4.8m high resolution nonuniform sampling Fourier transform spectrometer in the lower ultraviolet to near infrared range to systematically study the performance and differences of NUFFT method over resampling by interpolation FFT method by using the the interferograms of a 532nm solid-state multimode laser, a 960nm broad spectrum SLED as well as a 632.8nm Helium Neon laser reference source. This interferometer has a 4.8m scanning length thus allowing to study the NUFFT performance at very high resolutions.

Theory
Nonuniform discrete Fourier transform is the nonuniform analogy of regular discrete Fourier transform. It is defined as below by the following equation: where represents frequency at the n-th spectral point, represents the position of the k-th sampling point, ( ) represents the signal strength at the sampling position , and ( ) represents the spectral intensity at the frequency .
There are three types of nonuniform discrete Fourier transforms. The first type is when the sampling positions are distributed non-uniformly, the second type is when the spectral points are distributed non-uniformly, the third type is when both points are distributed non-uniformly. In fact, the discrete In order to achieve 4.8m scanning length, four retroreflectors is used to fold the optical paths at each arm by 8 times and we place the moving mirror of both arms onto the translation stage in back to back fashion to further increase optical path difference by two times and thus in result in 16 times increase in optical path differences in total. We use the simplest method to obtain the position of sampling points by adding a reference 632.8nm helium neon laser to the test light source by using a beam splitter.

Figure 1 schematics and pictures of our interferometer design
We use this interferometer the record the interferogram of a 532nm multimode solid state later and a 960nm broad spectrum SLED infrared light source as well as the 632.8nm Helium Neon laser reference source.
The various light sources used by this experiment is Thorlabs HRS015 frequency stabilized 632.8nm helium neon laser, Superlum SLD-MS-261 960nm SLED, CNI MGL-III-532-20mW solid state laser. The photodetector used here is Thorlabs PDA100A-EC 340nm-1100nm Switchable Gain Detector which outputs analog signal. The data acquisition device used is National Instrument NI-PCI-MIO-16E-4 with a maximum sampling rate of 1666666Hz. The translation stage used is a Physik Instrumente M-531.DDX translation stage with Mercury C-860.10 DC-motor controller.

Characteristics:
The non-constant moving speed of the translation stage of this interferometer makes it sample nonuniformly in the optical path difference domain. The following figure shows an example of the moving speed variation of the translation stage throughout the scanning length, the interferogram of the reference helium neon laser profile, the helium neon laser signal spectrum in time domain, and the spectrum of helium neon laser in optical path difference domain. The speed profile is obtained with the help of the helium neon reference laser. The method to determine the sampling point position is outlined below.

Figure 2 an example of the mirror translation speed profile
It can be seen from the example that the translation speed varies by as much as over 30 percent and the helium neon laser light is very close to ideal monochromatic source.

Method:
It is imperative to know the position of sampling points in nonuniform sampling. In this experiment a method based on the principle of the instantaneous phase of analytical signal of reference source is used to determine the position of sampling points.
For an ideal monochromatic light, the signal will simply be a cosine function with its phase being proportional to the optical path difference. Thus, the relative optical path difference can be easily determined by simply doing inverse cosine calculation of the reference signal.
However, in practice the amplitude of the signal will vary due to various reasons such as the divergence of the light beam as the mirrors move etcetera even though the reference light is assumed to be monochromatic. As can be seen from the example, the magnitude of the interferogram has a large variation in total and also many smaller local variations.
Thus, in this experiment a method based on analytical signal is used. Analytical signal is a complex value signal containing only the positive frequency part of the spectrum of the real signal. The spectrum of a real value signal will always be symmetric with equal negative and positive parts.
Thus, the analytical signal will always contain the same information as the real signal, and it can be obtained by inverse Fourier transformation of the positive part of spectrum.
The real signal of the helium neon laser in time domain in nonuniform sampling can be written as: where ( ) represents the signal, ( ) represents signal amplitude, ( ) represents the phase, represents time, represents position and represents the frequency of this monochromatic source. The analytical signal of this time domain real signal in nonuniform sampling can then be assumed to be as: where ( ) represents the complex analytical signal. The instantaneous phase can be assumed to be proportional to optical path difference in monochromatic source. Thus, the relative optical path difference can be determined if wavelength of the reference source is also known.
an example of the interferogram of the helium neon laser, the mirror moving of the interferometer, and the spectrum of the time domain signal of the helium neon reference laser.

Calculation
The Interpolation and FFT calculation are based on the SciPy python package using its interpolation module and FFT module.

Spectral profile comparison:
We first compare the spectrum of a 532nm solid state laser and a 960nm SLED from both methods. The interferogram of the 532nm laser and the 960nm SLED is also shown. It can be seen that the coherence length of the laser is very long while the coherence length of the SLED is very short.  It can be seen from the figure that the spectral profiles from both methods are visually undiscernible from each other. Thus, it can be said that the NUFFT method is at least as good as the interpolation method.

Spectral noise comparison:
Next we plot in log scale the spectrum of the 532nm laser interferogram which also contains the reference 632.8nm He-Ne laser source to compare the noise level in the calculated spectrum from both methods. Again we can see that their spectral noise profile is very similar to each other when the sampling rate is reasonable at around 1 sample per 100nm. The spectral noise level and shape from both methods are very similar. But it can also be seen that they are not the same exactly.

Impact of the 0 Herz component in signal to the spectral quality:
But there is one area where NUFFT and interpolation behaves very differently which is the influence of 0Hz or DC component in the signal to the level of noises in the resulting spectrum. Raw unfiltered interferogram will always contain a large amount of DC component due to the nature of interference. While in theory and in practice 0Hz component will not generate any spectral noises for interpolation based FFT or regular FFT method and can be simply ignored, this 0Hz component can in theory generate noises in nonuniform Fourier transform spectrum and our result on 960nm SLED shows that this indeed can sometimes have huge impact on the spectral quality as shown below. In general, this impact is less pronounced when the sampling rate is higher or when the spectral intensity is high as in the case of laser spectra but nonetheless this shows that it is important to remove the 0Hz component from signal in NUFFT method. The spectrum will become just as good as the interpolation ones once after removing the 0Hz component. While this problem may seem trivial as the DC component can be easily removed by subtracting the mean value of the interferogram, it shows that in practice it is very important to filter out 0Hz components first before NUFFT calculation as this source of error can be very tricky to identify if not known in advance. The reason for this behaviour here is that the 0Hz amplitude is relatively much larger compared to the amplitude of the interference pattern.

Spectral amplitude comparison:
We found that NUFFT method consistently produces higher spectral amplitudes than interpolation method.
According to the equation of nonuniform Fourier transform, for a monochromatic source the calculated spectral amplitude is equal to the signal amplitude times the number of sample points if the signal frequency happens to fall on one of the spectral points. Thus, according to theory NUFFT should be able to reproduce spectral intensity without loss of amplitude. The same can be said for the regular discrete Fourier transform because the discrete Fourier transform equation is only a special case of nonuniform Fourier transform equation. However, the same cannot be said for resampling by interpolation method because the interpolated signal will not be the same as the actual signal. This can result in lower signal amplitude and loss of amplitude in spectrum. One can expect that this loss would be dependent on sampling rates. Lower sampling rates will have more severe loss than higher sampling rates.
We found that NUFFT method consistently produces higher spectral amplitudes than interpolation method. According to the equation of nonuniform Fourier transform it should be able to reproduce spectral intensity without loss of amplitude while interpolation spectrum may lose some amplitude because the interpolated signal intensity is not the same as the actual ones.
We subtract the NUFFT spectrum of the 960nm SLED source with the interpolation FFT spectrum. It shows that NUFFT amplitude is consistently over 5% higher than interpolation with their difference having almost the same shape as the actual spectral profile. CubicSpline interpolation is also marginally higher than linear interpolation.

Figure 7 Magnitude difference of NUFFT spectrum and interpolation spectrum
The above result is obtained with a sampling rate of about 1 per 20nm which is fairly high for a 960nm signal. One can expect that the spectral amplitude by interpolation would be even lower with lower sampling rate. We compare the spectral profile of the 532nm laser at a sampling rate of about 1 per 210nm which is close to Nyquist rate. It can be seen that the interpolation has about 20 percent smaller amplitude.

Figure 8 spectral amplitude difference of NUFFT and interpolation in 532nm laser
Thus we have shown that interpolation would have lower and inconsistent spectral amplitudes. This may translate to potentially lower spectral signal to noise ratios as well as distorted spectral profile shapes with higher frequency part having lower amplitudes.

under-sampling and aliasing behaviour:
The periodic nature of regular discrete Fourier transform equation means that the calculated spectrum has a maximum limit in frequency and any frequency higher than the limit will fold back into the spectrum thus ruining the calculated spectrum, as can be seen from the equation. This problem is called aliasing.

( ) = ∑ ( ) 2 = ∑ ( ) 2 ( + )
Where + will be the false frequency appearing in the spectrum since + and so on is fundamentally undiscernible from in the equation.
Thus, not only the periodicity of the discrete Fourier transform places a maximum frequency limit on the spectrum but also will ruin it with aliasing.
On the contrary, nonuniform Fourier transform is inherently non periodic since the sample positions are randomly distributed and thus should have no maximum frequency limits and will also be immune to aliasing problem.
To test out this hypothesis, we under sample the 532nm laser by omitting 9 sample points out of every 10 from the interferogram and calculate the spectrum by NUFFT. We also plot the spectrum in log scale to compare the noise levels. It can been seen that the under sampled interferogram can perfectly reproduce the laser spectrum as well as the oversampled one but its noise level is significantly higher. It can also be seen here that the spectral resolution keeps the same because the resolution depends on the scanning length and not the number of sampling points. Interpolation FFT will also suffer from aliasing to some extent. Likewise, we under sample the helium neon reference laser to demonstrate the aliasing behaviour in interpolation FFT spectrum.

Figure 10 aliasing in interpolation FFT
Another issue of resampling by interpolation method is that the resampling rate is very arbitrary. It can be set to higher or lower than the actual average sampling rate. And the spectral result can be different depending on the chosen resampling rate especially in terms of aliasing behaviour because changing resampling rate would change period length. Generally, it can be expected that it is useless to reduce resampling rate but one may wonder what would happen if increasing resampling rate. We compare the behaviour of changing resampling rate with an under sampled helium neon laser profile. It shows that when resampling at higher rates it can recover the 632.8nm spectral line of the helium neon laser albeit significantly underestimating the spectral amplitude while resampling at average sampling rate would show aliasing problems.

Figure 11 changing resampling rates with an undersampled HeNe interferogram
Under sampling and anti-aliasing are two sides of same coin which is a result of the unique non periodic nature of the nonuniform Fourier transform equation. Here we not only demonstrate that NUFFT has unique advantage over interpolation, but also that nonuniform sampling itself has fundamental benefits over uniform sampling with its non-periodic nature and only NUFFT can fully utilise this potential.
Under sampling may not be useful when the coherence length is very short such as the SLED where too few number of sample points will result in too low signal to noise ratios but it certainly can be useful for long coherence sources such as the laser to significantly reduce sample size.

Non random electrical noise:
The photodetector and the data acquisition device will contain some electrical noises. The electric noise may not be completely random in reality. We block off all lights in the detector of interferometer and record the dark current electrical noise of it. It can be seen from the noise spectrum that it has some significant spikes. We test this property of both NUFFT and interpolation method. It shows that both methods can mitigate the intrinsic electric noise very well. The spikes are almost reduced to white noises.

Computation performance comparison in practice:
It is impractical to develop NUFFT or FFT software codes by users themselves because those are relatively complex algorithms and would take huge amount of time make. Thus, the availability of software packages is very important for practical purpose. The availability of NUFFT software is a weak point compared to interpolation method. FFT and interpolation are very mature mathematical problems and have standard packages in many programming languages. They probably have been available since the advent of computer era. But NUFFT is more of a niche product and there are usually no standard software packages to it. As a result, many existing NUFFT packages are made for only specific applications. This makes them not very usable for users from other fields.
Theoretically there should be no fundamental speed advantage or disadvantage of NUFFT since they speed up in similar ways. We compare the actual performance of our python implementation of NUFFT method implemented with FINUFFT python package and interpolation FFT method implemented with python SciPy interpolate and fft package in both computation time and memory consumption: 1. NUFFT is as fast as CubicSpline interpolation FFT in CPU computing time. The difference depends on sample size because the speed does not scale linearly with the number of sample points. It is about 32 second versus 48 second with a sample size of 40 million sample points.
2. NUFFT consumes much less memory than interpolation based FFT depending on the algorithm of interpolation, about 1/3 in linear interpolation and over 2/3 in CubicSpline interpolation.
3. There is another practical advantage by FINUFFT package in that it allows users to select only a subset of spectral points to calculate which significantly reduces computing time while standard FFT packages always compute whole spectrum.
Thus overall NUFFT is more efficient in our python case. Although the saving in time may be small compared to other processing procedures such as graph plotting etc.
FINUFFT is as easy to use as SciPy and NumPy packages. While it is entirely possible that this advantage of NUFFT is solely due to better optimization by FINUFFT, this at least demonstrates that NUFFT has practical advantages.

Discussion
The experiment contains many sources of noises. We use simulation to better study the noises due to data processing methods. Both NUFFT and interpolation method would introduce some noises into the spectrum. But their mechanism is completely different. For interpolation, this is due to that the interpolated signal is not exactly the same as actual ones while for NUFFT this is due to the intrinsic nature of the nonuniform Fourier transform equation itself. To better understand these spectral noise sources, we simulate an ideal monochromatic source at helium neon frequency with the sampling position determined from experimental data thus making the sampling simulation close to reality. We plot the simulated spectrum in log scale. It can be seen that CubicSpline interpolation is significantly better than linear interpolation. Also the noise level of NUFFT is comparable to interpolation but their noise shape is significantly different with the NUFFT one being much more irregular. We also test the under-sampling feature of NUFFT and show that under sampling will result in significantly higher noise levels. It can also be seen that linear interpolation is very terrible in terms of spectral amplitudes. And we also tested a variant of nonuniform Fourier transform equation by giving the signal a weigh based on the sampling rate. The idea is that the summation formula of nonstandard variant of nonuniform Fourier transform will converge to an integration formula when the sampling rate is high.
The simulation result shows that this nonstandard variant of NUFFT has significantly lower spectral noise level in the over sampled region. But it shows no improvement in under sampled region.

Conclusion
We have used experiment test to show that NUFFT method is as good as resampling by interpolation FFT method for nonuniform sampling Fourier transform Michelson interferometers method in spectral profile and spectral noise levels. It is also better in spectral amplitudes and has advantages in under sampling and anti-aliasing. Not only that, we have also shown that nonuniform sampling has fundamental benefits over uniform sampling due to its non-periodic nature with under sampling, anti-aliasing and electric noise mitigation properties. It also consumes less computer memory and is faster.
Thus we have shown that NUFFT is an equal or superior alternative to interpolation method for nonuniform sampling Fourier transform spectrometer. It can serve as an alternative method to help debug software codes which was the original motive that inspires us to try NUFFT in the first. And not only that, this paper also shows that nonuniform sampling has some unique benefits over uniform sampling and thus it is not only desirable when uniform sampling is infeasible but also counterintuitively can still be a better solution even if uniform sampling is available.