{'fraction_non_alphanumeric': 0.03533026113671275, 'fraction_numerical': 0.014592933947772658, 'mean_word_length': 5.325242718446602, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A Tool for Mining Large Correlation Tables : The Association Navigator

The Association Navigator is an interactive visualization tool for viewing large tables of correlations. The basic operation is zooming and panning of a table that is presented in graphical form, here called a “blockplot”. The tool is really a tool box that includes, among other things: (1) display of p-values and missing value patterns in addition to correlations, (2) mark-up facilities to highlight variables and sub-tables as landmarks when navigating the larger table, (3) histograms/barcharts, scatterplots and scatterplot matrices as “lenses” into the distributions of variables and variable pairs, (4) thresholding of correlations and pvalues to show only strong and highly significant p-values, (5) trimming of extreme values of the variables for robustness, (6) “reference variables” that stay in sight at all times, and (7) wholesale adjustment of groups of variables for other variables. The tool has been applied to data with nearly 2,000 variables and associated tables approaching a size of 2,000×2,000. The usefulness of the tool is less in beholding gigantic tables in their entirety and more in searching for interesting association patterns by navigating manageable but numerous and interconnected sub-tables.