{'fraction_non_alphanumeric': 0.023386342376052385, 'fraction_numerical': 0.0, 'mean_word_length': 5.445783132530121, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
ESTIMATION OF CLASSIFIER ' S PERFORMANCE WITH ERROR COUNTING

Because of the complexity of error estimation in statistical pattern recognition, error counting methodology has been extensively used for evaluating the performance of a classifier. The paper compares different error estimation procedures, both theoretically and experimentally by simulation. The classical error counting methods, like resubstitution, hold-out, leave-one-out and bootstrapping, are compared with the risk averaging methods and with methods using the relationships of error and reject tradeoffs. These second generation estimators have two clear advantages. First, they are designed to have a smaller variance. Secondly, and more importantly, the test samples can be unclassified. Thus, the risk -averaging methods are economical and in addition the mislabelling problems in the test set is avoided. Finally they are well suited in non-parametric methods and thus provide a possible solution for a general classification system. The simulation studies show that this is possible to achieve.