{'fraction_non_alphanumeric': 0.021709633649932156, 'fraction_numerical': 0.0027137042062415195, 'mean_word_length': 5.254237288135593, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Transform-Based Computation of the Distribution of a Linear Combination of Random Variables Over Arbitrary Finite Fields

Several authors have developed a Hadamard (variously called FFT) transform technique for fast belief propagation over GF(q), where in all prior work q=2m for some m. The belief propagation step which employs the transform generalizes Gallager's lemma for computing the distribution of a sum of variables over a finite field. In this paper, the limitation to fields of characteristic 2 is eliminated, so that computations can take place over finite fields of arbitrary prime characteristic. This opens the door, for example, for BP decoding algorithms over the same fields that are used for algebraic geometry codes.