{'fraction_non_alphanumeric': 0.02242152466367713, 'fraction_numerical': 0.0, 'mean_word_length': 5.471014492753623, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Data-Intensive Computation and the Datacentric Grid

The analysis of large datasets has become an important tool in understanding complex systems in areas such as economics and business, science, and engineering. Such datasets are often collected in a geographically distributed way, and cannot, in practise, be gathered into a single repository. Applications that work with such datasets, for example data-mining algorithms, cannot control most aspects of the data’s partitioning or arrangement. As a consequence, both new architectures and new algorithms are needed. This paper presents the design of the DCGrid, a datacentric grid, and describes some parallel and distributed algorithms for data mining that are particularly suited to large-scale distributed datasets. Some of these algorithms permit genuine superlinear speedup so they are effective even for modest levels of parallelism.