{'fraction_non_alphanumeric': 0.01928374655647383, 'fraction_numerical': 0.0, 'mean_word_length': 5.267241379310345, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Noise equivalent dimensions in eigenspaces

Eigenspaces are commonly used for dimensionality reduction, either for a compact encoding or for classification of high dimensional data. However in the presence of noise, the eigenspaces that were created using low-noise data, no longer explain the noise-corrupted data. In this paper, we present the notion of noise equivalent dimensions (NED) as a means of increasing the contribution of signal strength to compensate for the contribution of noise. Although the notion of NED is created with reconstruction error in mind, the additional dimensions may very well be used for classification purposes. Experiments with synthetic and real data are presented to demonstrate their use.