{'fraction_non_alphanumeric': 0.0166865315852205, 'fraction_numerical': 0.0023837902264600714, 'mean_word_length': 4.714285714285714, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Determining the equivalence of algebraic expressions by hash coding

In algebraic manipulation one often wants to know if two expressions are equivalent under the algebraic and trigonometric identities. One way to check this is to substitute a random value for each variable in the expressions and then see if both expressions evaluate to the same result. Round-off and overflow errors can be avoided if the evaluation is done modulo a large prime number. Of course, there is still a small probability of a random match. If arithmetic expressions in exponents are evaluated, using the same prime number then identities such as xl/2 ï¾ xl/2 &equil; x will not necessarily hold. However, by proper choice of the prime number and the use of special case checks these identities as well as many trigonometric identities can often be preserved.