{'fraction_non_alphanumeric': 0.031513787281935844, 'fraction_numerical': 0.005064715813168261, 'mean_word_length': 4.887417218543046, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Uniform, Versatile and Efficient Dense and Sparse Multi-Dimensional Arrays

We describe a Java toolkit for rectangular dense and sparse multi-dimensional arrays. Such a toolkit should be usable, efficient and allow for the definition of many algorithms from disparate domains. Balancing these competing requirements is challenging. We address this issue by providing a few non-overlapping general-purpose concepts that a user can combine in a straightforward, uniform and efficient manner to generate complex behaviour. The most important aspect of this approach is that a single polymorphic interface provides uniform access to any matrix or view thereof. Every matrix and view class implements the same interface with exactly the same semantics, no more, no less. Therefore, everything that can be done with a dense matrix can without code change also be done with a sparse matrix, and vice-versa. Everything that can be done with a normal matrix (dense or sparse) can without code change also be done with any (abitrarily nested) matrix view, and vice-versa. We describe non-delegating true matrix views for sub-ranging, striding, transposition, slicing, index flipping, cell selection, as well as sorting, permuting and partitioning. We also describe a space and time efficient sparse representation based on hashing. We then compare the performance with three other toolkits, including C/C++. Throughputs up to 500 MB/s and 100 Mflops/s are demonstrated in the dense case, and normalized up to 24 GB/s and 3 Gflops/s in the sparse case. In every benchmark, our toolkit is among the leading two. Performance is better than C/C++. Our overall conclusion is that rich and complex behaviour cannot only be provided with minimal usability impact, but also with high efficiency.