{'fraction_non_alphanumeric': 0.02062780269058296, 'fraction_numerical': 0.003587443946188341, 'mean_word_length': 5.413793103448276, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
An adaptive incremental LBG for vector quantization.

This study presents a new vector quantization method that generates codewords incrementally. New codewords are inserted in regions of the input vector space where the distortion error is highest until the desired number of codewords (or a distortion error threshold) is achieved. Adoption of the adaptive distance function greatly increases the proposed method's performance. During the incremental process, a removal-insertion technique is used to fine-tune the codebook to make the proposed method independent of initial conditions. The proposed method works better than some recently published efficient algorithms such as Enhanced LBG (Patane, & Russo, 2001) for traditional tasks: with fixed number of codewords, to find a suitable codebook to minimize distortion error. The proposed method can also be used for new tasks that are insoluble using traditional methods: with fixed distortion error, to minimize the number of codewords and find a suitable codebook. Experiments for some image compression problems indicate that the proposed method works well.