{'fraction_non_alphanumeric': 0.014341590612777053, 'fraction_numerical': 0.0, 'mean_word_length': 5.2145748987854255, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Geometric estimation of the inherent dimensionality of a single material cluster in multi- and hyperspectral imagery

The inherent dimensionality of a spectral image can be estimated in a number of ways, primarily based on statistical measures of the data cloud in the hyperspace. Methods using the eigenvalues from a Principal Components Analysis, a Minimum Noise Fraction transformation, or the Virtual Dimensionality algorithm are widely used as applied to entire images typically with the goal of reducing the dimensionality of an image in its entirety. However, it is desirable to understand the dimensionality of individual components within a hyperspectral scene, as there is no a priori reason to expect all distinct material classes in the scene to have the same inherent dimensionality. Additionally, in complex scenes containing non-natural materials, the lack of multivariate normality of the data set implies that a statistically based estimation is less than optimal. Here, a geometric approach is developed based on the local estimation of dimensionality in the native data hyperspace. It will be shown that the dimensionality of a collection of data points (k) in the full n dimensions (where n is the number of spectral channels measured) can be estimated by calculating the change in point density as a function of distance in the full n dimensional hyperspace. Simple simulated examples to demonstrate the concept will be shown, as well as applications to real hyperspectral imagery collected with the HyMAP sensor.