{'fraction_non_alphanumeric': 0.0192131747483989, 'fraction_numerical': 0.007319304666056725, 'mean_word_length': 5.590361445783133, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Script recognition using inhomogeneous P2DHMM and hierarchical search space reduction

Pseudo-2D hidden Markov models (P2DHMMs) manage the assignment of observations to states quite well, whereas the calculation of probabilities is a great problem. To overcome some of the defects of P2DHMMs, we propose the inhomogeneous P2DHMM (IP2DHMM). In contrast to other approaches, it is consistently able to model state duration and observation matrices of fixed height at no additional cost in computational load and number of parameters. At the same time, it enables an efficient pruning mechanism and allows a considerable reduction in decoding time. In order to further reduce the computational load, a combination scheme is presented for multiple recognition stages, using lexical tree beam searching at different resolutions and with models of different complexities (IP2DHMM and 1DHMM). The resulting system is evaluated on three different databases from different application fields: cursive script recognition, recognition of German legal amounts and recognition of city names from live mail.