{'fraction_non_alphanumeric': 0.017839444995044598, 'fraction_numerical': 0.004955401387512388, 'mean_word_length': 5.352201257861635, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
From characters to words: dynamical segmentation and predictive neural networks

We present the extension of a neural predictive system primitively designed for on-line character recognition to words. Feature extraction is performed after resampling the pen trajectory information, recorded by a digitizing tablet. Each word is modeled by the natural concatenation of letter-models corresponding to the letters composing it. Successive parts of a word trajectory are this way modeled by different neural networks and only transitions from each one to itself or to its right neighbors are permitted. A holistic and dynamical segmentation allows one to adjust letter-models to the great variability of handwriting encountered in the words. Our system combines multilayer neural networks and dynamic programming with an underlying left-right hidden Markov model (HMM). Training was performed on 7000 words from 9 writers, leading to good results in the letter-labelling process, without using any language model.