{'fraction_non_alphanumeric': 0.018650088809946713, 'fraction_numerical': 0.0008880994671403197, 'mean_word_length': 5.590643274853801, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Increasing the Reliability of Full Text Documents Based on the Use of Mechanisms for Extraction of Statistical and Semantic Links of Elements

Approaches, principles and methods for increasing the reliability of information in electronic document management systems based on the use of statistical and semantic information redundancy have been developed. Methods are proposed, including mechanisms of statistical coding, extraction of statistical and semantic links, binary modular summations, and parity control. The probabilities of occurrence of single, double and t -multiple errors in texts, the required amount of information redundancy, the size of additional characters included in the information and control bits of the coding method are determined. A technique is proposed for optimizing the reliability of information based on mechanisms for coding, summing, checking, extracting statistical and semantic links of elements, as well as regulating the length of control bits. A software package for detecting and correcting one, two and t - multiple errors in the Sphinx-4 framework environment has been implemented.