{'fraction_non_alphanumeric': 0.014763779527559055, 'fraction_numerical': 0.0, 'mean_word_length': 4.878612716763006, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Determining the Dimensionality of Hyperspectral Imagery

There are a number of reasons why reduction of large data sets is necessary, for example the amount of data may be too large for some data mining programs. Something the amount of data may exceed the processing capability of a program, as it is usual in the case of hyperspectral images. The data has generally a large number of variables to analyze, some of which have more input than others. It is often necessary to carry out an analysis to know which variables to select. Several methods exist to determine how many variables to select among which we have: variation total, CSD, size of variances of principal components, Scree test, and the rank of a matrix criterion. In this paper we will make a comparison between these criteria with the purpose of determining which method present the best data set reduction on hyperspectral image classification, preliminary results show that the variation total criteria reduces drastically the data set dimension.