{'fraction_non_alphanumeric': 0.014848289218850872, 'fraction_numerical': 0.0, 'mean_word_length': 5.798245614035087, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Statistical Aspects of Near-Infrared Spectroscopy for the Characterization of Errors and Model Building

Due to the complex nature of near-infrared (NIR) spectra, it is usually very difficult to provide quantitative interpretations of spectral data. As a consequence, careful building and validation of calibration models are of fundamental importance prior to development of useful applications of NIR technologies. For this reason, this work presents a statistical study about the NIR spectroscopy, analyzing the NIR behavior when the experimental conditions are changed. Near-infrared spectra were measured at different temperatures and stirring velocities for systems containing a pure solvent and a suspension of polymer powder in order to perform the error analysis. Then, mixtures of xylene and toluene were analyzed through NIR at different temperatures and stirring velocities and the obtained data were used to build calibration models with multivariate techniques. The results showed that the precision of the NIR measurements depends on the analytical conditions and that unavoidable fluctuations of spectral data (or spectral data variability) are strongly correlated, leading to full covariance matrices of spectral fluctuations, which has been surprisingly neglected during quantitative analyses. In particular, modeling of the xylene/toluene NIR data performed with different multivariate techniques revealed that the principal directions are not preserved when the real covariance matrix of measurement errors is taken into account.