{'fraction_non_alphanumeric': 0.01963439404197698, 'fraction_numerical': 0.006770480704129994, 'mean_word_length': 4.888446215139442, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Automatic flat field algorithm for hyperspectral image calibration

Image spectra calibration is of great importance for further processing and feature extraction. In this paper, an automated flat field reflectance calibration algorithm (AFFT) is proposed. This algorithm is an improvement to the traditional flat field transformation calibration. It is based on the fact that the so-called flat field is a flat block of high brightness and relative flat spectral response, and at a certain wavelength range (.e.g. 500-700nm) the brightness or radiance of the flat field is a certain multiple of the average spectrum of the image. Because the average image spectrum spectrum usuall is relatively flat, so a certain multiple of the average spectrum can be regarded as the criterion (or threshold) to select flat field pixels. So such parameters as wavelength range, multiple increment between flat field and the average image spectrum and number of the largest area block are set to determine the useful flat field so that an average spectrum of the flat field is obtained. By using this flat field spectrum as solar/atmospheric response, hyperspectral image can be calibrated to reflectance image. In the end, AFFT was validated by one PHI image acquired in Japan, 2000. It turns out that AFFT is effective to search all the flat fields which meet the fixed terms automatically and promptly, the spectra transformed by this method are much smoother and reliable to some extent.