{'fraction_non_alphanumeric': 0.012850467289719626, 'fraction_numerical': 0.0, 'mean_word_length': 5.035211267605634, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
An adaptive data compression method based on context sorting

Every symbol in the data can be predicted by taking its immediately preceding symbols, or context, into account. This paper proposes a new adaptive data compression method based on a technique of context sorting. The aim of context sorting is to sort a set of contexts in order to find previous contexts similar to the current one. The proposed method predicts the next symbol by ranking the previous context-symbol pairs in order of context similarity. The codeword for the next symbol represents the rank of the symbol in this ordered sequence. The compression performance is evaluated both analytically and empirically. Although the proposed method uses no probability distribution to make a prediction, it has comparable compression performance to the best known data compression utilities.