{'fraction_non_alphanumeric': 0.037243947858473, 'fraction_numerical': 0.0, 'mean_word_length': 4.429292929292929, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Probabilistic Sequence Models

On the left is pictured a model that repeatedly prints a letter according to fixed emission probabilities. Such a model (perhaps with different letters and/or probabilities) is called an i.i.d. (independent identically distributed) model. With the model pictured on the right, each state (the conventional name for nodes or vertices in this context) emits a fixed letter, and the probabilistic nature comes from the transition probabilities, which control the frequency with which the next state is chosen at each step. Here, for instance, the model emits runs of A and of B, with the former generally longer than the latter. A model of this sort, i.e., where each state emits a fixed letter and where no two states have the same letter, is called a Markov chain. It is straightforward to see that Markov models are more general than i.i.d. models, i.e., for any i.i.d. model there is an equivalent Markov model, but not necessarily vice versa. There are a number of additional kinds of probabilistic sequence models, as we will soon discover.