{'fraction_non_alphanumeric': 0.023743016759776536, 'fraction_numerical': 0.0, 'mean_word_length': 4.690476190476191, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Discovering roll-up dependencies

We introduce the problem of discovering functional determinacies that result from “rolling up” data to a higher abstraction level. Such a determinacy is called a Roll-Up Dependency (RUD). An example RUD is: The probability that two files in the same directory have the same file extension, is greater than a specific number. We show the applicability of RUDs for OLAP and data mining. We consider the problem of mining RUDs that satisfy specified support and confidence thresholds. This problem is NP-hard in the number of attributes. We give an algorithm for this problem. Experimental results show that the algorithm uses linear time in the number of tuples of the input database.