{'fraction_non_alphanumeric': 0.020737327188940093, 'fraction_numerical': 0.0015360983102918587, 'mean_word_length': 4.949771689497717, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Empirically Minimax Affine Mineralogy Estimates from Fourier Transform Infrared Spectrometry Using a Decimated Wavelet Basis

The Fourier transform infrared (FT-IR) spectrum of a rock contains information about its constituent minerals. Using the wavelet transform, we roughly separate the mineralogical information in the FT-IR spectrum from the noise, using an extensive set of training data for which the true mineralogy is known. We ignore wavelet coefficients that vary too much among repeated measurements on rocks with the same mineralogy, since these are likely to reflect analytical noise. We also ignore those that vary too little across the entire training set, since they do not help to discriminate among minerals. We use the remaining wavelet coefficients as the data for the problem of estimating mineralogy from FT-IR data. For each mineral of interest, we construct an affine estimator ◯ of the mass fraction x of the mineral of the form , where is a vector, is the vector of retained wavelet coefficients, and b is a scalar. We find and b by minimizing the maximum error of the estimator over the training set. When applying the estimator, we “truncate” to keep the estimated mineralogy between 0 and 1. The estimators typically perform better than weighted nonnegative least-squares.