{'fraction_non_alphanumeric': 0.011615628299894404, 'fraction_numerical': 0.0031678986272439284, 'mean_word_length': 5.771428571428571, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Significance-weighted feature extraction from hyperdimensional data

Extracting significant features is essential for processing and transmission of a vast volume of hyper-dimensional data. Conventional ways of extracting features are not always satisfactory for this kind of data in terms of optimality and computation time. Here we present a successive feature extraction method designed for significance-weighted supervised classification. After all the data are orthogonalized and reduced by principal component analysis, a set of appropriate features for prescribed purpose is extracted as linear combinations of the reduced components. The method was applied to 500 dimensional hyperspectral data which were obtained from tree leaves of five categories. Features were successively extracted, and they were found to yield more than several percents higher accuracy for the classification of prescribed classes than a conventional method does.