{'fraction_non_alphanumeric': 0.01440922190201729, 'fraction_numerical': 0.0, 'mean_word_length': 5.0964912280701755, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Estimating the total model error from a standard validation test

The problem of assessing the quality of a given, or estimated model is a central issue in system identification. Classical model validation procedures are considered. We discuss the principles by which we reach confidence in a model through such validation techniques, and also how the distance to a "true" description can be estimated this way. In particular we stress how the typical model validation procedure gives a direct measure of the model error of the model test, without referring to its ensemble properties. Several model error bounds are developed for various assumptions about the disturbances entering the system.