{'fraction_non_alphanumeric': 0.01877346683354193, 'fraction_numerical': 0.0025031289111389237, 'mean_word_length': 5.106870229007634, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A New Algorithm for Calculating Adaptive Character Wordlength via Estimating Compressed File Size

In statistical compression algorithms, character word length of 8 bit is being used. Using an optimum character word length, b, where b>8, will improve compression ratio. There are different ways to calculate optimum character word length. In this paper we introduce a new algorithm named Estimated Compressed File Size (ECF) to find optimum character word length by estimating compressed file size. In this new algorithm, different metrics such as header file size and entropy are effective to determine optimum character word length. The ECF algorithm is a good lower bound for estimating compressed file size. ECF is a rapid algorithm for large files and does not depend to statistical algorithms.