{'fraction_non_alphanumeric': 0.008484848484848486, 'fraction_numerical': 0.0, 'mean_word_length': 5.883333333333334, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
THE RELIABILITY OF EXPERT ASSESSMENTS DURING EXAMINATION OF SCIENTIFIC, TECHNOLOGICAL AND INNOVATIVE PROJECTS IN THE INFORMATION SYSTEM OF THE FEDERAL ROSTER OF EXPERTS

The question of the reliability of the results of examination of scientific, technological and innovative projects conducted using the Information System of the Federal Roster of Experts of Scientific and Technological Spheres is considered. Typical examination schemes with different composition of expert groups are analyzed, sources and causes of expert errors are identified. A method for quantifying the reliability of the final expert assessments for various examination schemes based on a probabilistic approach is proposed. Recommendations are given on improving expert survey procedures in order to increase the reliability of expert assessments.