{'fraction_non_alphanumeric': 0.023785226469895598, 'fraction_numerical': 0.0043959494465813645, 'mean_word_length': 5.089866156787763, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
FREQUENCY RESPONSE OF NORRIS GAP DERIVATIVES AND ITS PROSPERITIES FOR GAS SPECTRA ANALYSIS

The article deals with an analysis of the properties of Norris gap derivatives. It discusses issues related to determining information from optical spectra measured with spectrometers. Impulse responses of differentiating filters were introduced using both Norris and Savitzky-Golay methods. The amplitude-frequency responses of the first and second order Norris differentiating filters were compared. The length impact of both segment and gaps on the frequency characteristics of filters was compared. The processing of exemplary gas spectra using the discussed technique was subsequently presented. The effect of first and second order derivatives on the spectra of carbon monoxide rotational lines for low resolution measurements is investigated. The Norris method of derivatives are very simple to implement and the calculation of their parameters does not require the use of advanced numerical methods.


Introduction
Spectroscopy is an analytical method finding increasing use in various fields, as it provides a quick and cheap alternative to many other types of techniques. Information are obtained as a result of the analysis of radiation, which interacts with the tested sample mainly by transmittance, reflectance or radiation emitted by the sample [16]. In the field of spectroscopy, in addition to the continuous development of measuring equipment, new methods of data analysis are essential. Data processing algorithms should be adapted to the type of analyzed signals and information to be obtained. New produced devices have better resolution which allows additional information to be obtained. Unfortunately, increasing the dimensions of the data sometimes makes it difficult to interpret. The type of processed data (qualitative or quantitative) also changes the approach to the algorithms used. The first step in determining information is data acquisition. After that, there is the data processing pipeline where very different computational methods may be utilised. Before starting to develop an algorithm for the analysis of specific spectra, the type of data and their complexity should be defined. Including what features can be determined and what distortions the signal contains.
The measured spectrum contains from several hundred to several thousand variables. In addition to substantial information, the spectra also contain other types of artifacts. The measured spectrum always contains an unwanted distorted background continuum signal, interference and noise. We understand interferences to be components having a shape with a specific structure. Background usually has a shape slowly varying with the wavelength. The typical shape of a baseline is [13]: offset, slope and curvature. There are many preprocessing methods to improve the spectral signal, extract some features and reduce the noninformative components. In the case of background correction for a single spectral line, the simplest method is three-point correction which assumes a parabolic type distortion [18]. More advanced methods use a specific spectral window. Heuristic methods are also often used, the result of which depends most on the experience of the experimenter. For some types of spectra, the best properties are obtained by means of derivatives. The first derivative causes elimination of a background constant value and the second derivative suppresses a linear varying background. So derivatives are able to significantly reduce continuous background. Additionally, derivatives increase the spectral resolution and improve the selectivity of the analysis. The second derivative of the typical spectral lines (Gauss or Lorentz) has a narrower shape than the original lines. Derivative filters belong to feature extraction algorithms, which consist in determining new signal features. Unfortunately, the differentiation operation reduces the signal-to-noise ratio and is therefore often related to smoothing of data, wherein, the smoothing is performed before or simultaneously with the derivative. The effect of achieving a well-smoothed derivative requires designing a low-pass filter with a low cut-off frequency [7].
In spectroscopy, derivatives are performed mainly by two methods: by a simple point difference and by the Savitzky-Golay method. Savitzky-Golay (SG) filters have been proposed for the analysis of chemical data [17]. Many modifications of the original SG method were made, and new versions of filters with various properties are proposed even recently [2,3,11]. It should be noted that the SG filters are not optimal and their frequency response is not ideal. SG filters are characterized by low computational complexity when used. However, determining their coefficients requires the use of tables or appropriate computer programs. This article will introduce the properties of the little-known Norris gap derivative method [15]. The analysis will be carried out from the signal processing and spectroscopic data analysis perspective. The considered method has proven positive properties as a preprocessing method for PLS model calibration [20]. The gap derivatives are shortly determined as an approximation of an analytical derivative by simple finite difference [5]. The frequency response of Norris derivative filters with smoothing will be derived in the next paragraphs. The effect of the parameters on the shape of the main-lobe and side-lobe will be determined. An exemplary optical spectrum containing rotational lines of gases will also be analyzed.

Savitzky-Golay and Norris derivative
The first derivative represents the rate of change of function. Similarly, the second derivative measures the value of change of the first derivative. In the digital world, "derivative" means calculating the differences between some wavelength. The simples' difference of adjacent wavenumbers is very sensitive (amplifies) to high frequency noise, less more mid-frequency and produces high attenuation of low-frequency noise. In the simplest version, the difference is calculated between adjacent points on the spectrum. Simple forward difference: ( ) = ( ( + ℎ) − ( ))/ℎ (1) But the difference may by calculated between two spectrum points which are separated by two or more points. The separation width is called a gap [4]. Increasing the gap can severely change the derivative shape and in practice can be applied only in very slowly changing spectra. A first-order gap derivative is given be following equation [5]: A second order gap derivative is: The frequency responses in Figure 1 have been scaled so that their shapes can be compared. Unscaled characteristics are those for the simple forward difference. The frequency has been normalized to the Nyquist frequency which is half of the sampling frequency.
The Norris derivative consists of calculation of the difference between two values which are averages of two adjacent points [9]. Further, the number of averages points, called segments, can be increased. At the same time, the gap between particular segments can be changed [10]. The bigger the gap, the broader the shape of the results with additional stronger noise suppression.
Example impulse response of Norris first derivative filters [9]: The theoretical basis of the Savitzky-Golay method, including derivative filters, consist of a least-square polynomial approximation of a curve between a section of data. The derivative is a slope of the curve in the central point of the data window. The SG differentiation filter is a finite impulse filter described by two parameters: the order of the polynomial of approximation, and the size of the data window. The filter coefficients are integers with an appropriate scaling factor [14].

Frequency response of Norris derivative filters
Spectrometers tends to produce drift related to instrumental and environmental factors which are fluctuations of the radiation source, detector sensitivity, temperature and sample changes. Unfortunately, drift is not independent in nature and cannot be eliminated even by multivariate calibration models [1]. Derivative methods can be used for this purpose. At the same time, the noise that appears at higher frequencies should also be taken into account. To analyze such problems in signal processing, frequency response characteristics are used. The shape of the main-lobe and the level of the side-lobes indicate the properties of a particular filter. However, they are very rarely analyzed in the processing of spectroscopic data. The most frequently analyzed are the shapes of specific spectra that have been processed using a given filter.
The frequency response of an ideal first order differentiator is a line with some slope what gives some kind of high-pass filter H(e j ) = j. To attenuate the high frequency, additional low-pass filtering is applied. As a result, we get a low-pass differential filter [19]. The ideal frequency limited differentiator has a frequency response that equals zero for a high frequency. As can be seen in Figure 2, this property in Norris filters can be observed by using a gap (central difference operator). But only the use of segments 3 and 5 effectively eliminates the higher frequencies (Fig. 3, 5). For efficient filtering of high frequencies, a long filter response is required, e.g. 5 segments and 5 gaps.   Similar to first order analysis can be performed for second order derivative. The ideal frequency response of the filters is increased square as a function of frequency  2 with a constant π phase. Figure 4 presents the frequency response for segment = 1 and with three value of gap. Figures 5 and 6 reflect the impact of increasing the segment length as a responsible of low-pass filtering effect.
It is very important to mention that normalization for a Norris derivative is performed only for segments and not for the convolution interval length [9].Therefore, the ratio of the side lobe to the main lobe, and not its actual value, should be compared.

Gas spectra processing by Norris derivative
The derivative eliminates constant baseline (offset) variation, and the second derivative eliminates the baseline linear trend. Additionally, the derivative enhances weaker absorption lines and is able to resolve closely spaced bands. The second derivative of a symmetric band also gives symmetric and negative shapes. But for asymmetric bands, the second derivative peaks are shifted. In practice, before calculating derivative spectra proper selection of algorithm parameters is required. The basic information here can be the shape of the original measured spectrum [12].   By application of a segment length of 3 or 5 (Figure 8 and 9), the processed spectral lines are very smooth and rounded. Applying wider segments leads to visible noise elimination and stronger regularity of the obtained shapes. Obtaining optimal parameters of differentiation requires analysis of its effect on zero order spectra. It should also be noted that in the spectra of the second derivative, a high side lobe levels appears, which can interfere with other closely spaced neighbored rotational lines.

Summary
The article presents the issue of the pre-processing of spectrometric data in order to eliminate undesired artifacts. One of the most frequently used methods for this purpose is the use of derivatives. In practice, most frequently, first and second order derivatives are used. They can be calculated with simple differences and with more sophisticated methods such as Savitzky-Golay. The Norris method is simpler than the Savitzky-Golay algorithm and uses two parameters: gap and segment. Gap increases the length of the window on which the derivative is executed and is responsible for the differentiation properties. The segment is the length of the signal fragment that is averaged and is therefore responsible for the low-pass properties of the filter. The length of the filter impulse response depends on the gap and segment.
The article presents the frequency response characteristics of Norris filters for the combination of gap values equaling 1, 3 and 5; and segments equaling 1, 3 and 5. Their frequency responses are not perfect, but with such a relatively short impulse response length, they can be considered to be correct. It can be observed that by increasing both the gap and segment size, a better signal-to-noise ratio is achieved.
It should be noted that, in practice, the selection of the derivative method is performed experimentally for each type of data by performing calculations for several filter parameters. By performing calculations for several values of gap and segments, their optimal Norris derivative can be quickly selected.