{'fraction_non_alphanumeric': 0.018011527377521614, 'fraction_numerical': 0.008645533141210375, 'mean_word_length': 5.256756756756757, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
[A simple and convenient standardization algorithm of NIR spectra].

Due to the limitation of current algorithms for NIR spectral analysis model transfer, a simple and convenient algorithm to standardize the spectra was proposed, and a new performance index called spectra standard error (SSE) was also constructed to evaluate the validity of model transfer algorithms. SSE expresses the ratio of J2 to J1, where J2 describes the distances between the spectra of the same sample using different instruments, and J2 describes the average distance between the spectra of different samples using the original instrument for their central spectrum. In the present paper we first used Savitzky-Golay smoothing to realize baseline correction for different spectra, and then applied standard normal variate method to standardize spectra and polynomial filtering to avoid noise. Besides, we optimized the wavelength range and the window width in Savitzky-Golay smoothing in order to minimize the SSE. After these steps, the standardized spectra can be applied to spectral analysis modeling. By the new algorithm, neither collecting a great number of samples nor need measuring all spectra of training samples using different instruments are needed. For a set of gasoline samples, SSE can be reduced from 1.418 to 0.167 via the new standardization algorithm and satisfactory results were obtained.