{'fraction_non_alphanumeric': 0.012357414448669201, 'fraction_numerical': 0.0, 'mean_word_length': 5.58125, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Compact and Fast Algorithms for Regular Expression Search

This paper describes an improvement of the brute force determinization algorithm in the case of homogeneous NFAs, as well as its application to pattern matching. Brute force determinization with limited memory may provide a partially determinized automaton, but its bounded complexity makes it be a fail-safe procedure contrary to the classical subset construction. We investigate the particular case of pattern matching based on Glushkov NFAs, which involves the determinization of an almost homogeneous automaton. Actually, our algorithm is inspired both by recent results of Champarnaud concerning the subset automaton of a homogeneous NFA and by the algorithm recently designed by Navarro and Raffinot to implement the brute force determinization of the Glushkov NFA of a regular pattern. We obtain an algorithm with an average exponentially smaller memory requirement for a given level of determinization, which, considering a bounded memory, implies a quadratically smaller parsing time.