{'fraction_non_alphanumeric': 0.01627906976744186, 'fraction_numerical': 0.0, 'mean_word_length': 4.949308755760368, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Multilevel Linear Dimensionality Reduction using Hypergraphs for Data Analysis

Classical algorithms used for dimension reduction can be time-consuming when the data set is large. In this paper we consider a method based on hypergraph coarsening to find a smaller set of data representing a given data set, prior to performing the projection into the low-dimensional space. The cost of the dimensionality reduction process is reduced because of this hypergraph-based pre-processing step. In a hypergraph model, each data item is represented as a vertex and related data items are connected by a hyperedge, which is simply a subset of vertices. To coarsen the data, we use a method that merges pairs of vertices. In the multilevel framework, the coarsening is recursively repeated until a coarsened data set of a certain size is reached. Then we project the coarsened data into a lower dimensional space, using a known linear dimensionality reduction method. The same linear mapping from the coarsened data is then applied to the original data set for projecting data into lowdimensional space. As an application of this idea, we consider text mining. Experimental results indicate that the multilevel hypergraph technique proposed in this paper offer a very appealing cost to quality ratio.