{'fraction_non_alphanumeric': 0.019653179190751446, 'fraction_numerical': 0.009248554913294798, 'mean_word_length': 5.6106870229007635, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A reduced ambiguity lexical system.

Natural human languages have proven to be sub-optimal in artificial intelligence applications because of their tendency to inexact representation of meaning. The author has devised a technique for converting human language to and from a compact byte-coded intermediate representation, which is processed more easily by computer systems. A specialized lexical engine based on IEEE Standard 1275-1994 was created to embed redundant information invisibly within the byte-coded text stream, to enable use of a variety of alphabets, grammars, and pronunciation rules (including slang and regional dialects). Very large vocabularies in a variety of human languages are supported. These lexical tools are designed to facilitate speech recognition and speech synthesis subsystems, universal translators and machine intelligence systems.