{'fraction_non_alphanumeric': 0.016891891891891893, 'fraction_numerical': 0.0, 'mean_word_length': 5.732954545454546, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Performance Analysis of Content-Based Identification Using Constrained List-Based Decoding

This paper is dedicated to the performance analysis of content-based identification using binary fingerprints and constrained list-based decoding. We formulate content-based identification as a multiple hypothesis test and develop analytical models of its performance in terms of probabilities of correct detection/miss and false acceptance for a class of statistical models, which captures the correlation between elements of either the content or its extracted features. Furthermore, in order to determine the block/codeword length impact on the identification's accuracy, we analyze exponents of these probabilities of errors. Finally, we develop a probabilistic model, justifying the accuracy of identification based on list decoding by evaluating the position of the queried entry on the output list. The obtained results make it possible to characterize the performance of traditional unique decoding, based on the maximum likelihood for the situations when the decoder fails to produce the correct index. This paper also contains experimental results that confirm theoretical findings.