{'fraction_non_alphanumeric': 0.025559105431309903, 'fraction_numerical': 0.0, 'mean_word_length': 5.526041666666667, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Multi Head Self Attention Mechanism Based Multi-scale Spatial-Temporal Mixed Text Classification Method Research

In text classification tasks, convolutional method can extract local features of text, but it ignores context semantic information and long-distance dependency. The mechanism of long-term and short-term memory can make up for this shortcoming, but it cannot extract local features of text effectively. Therefore, this paper proposes a Multi Head self Attention mechanism based Multi-scale Spatial- Temporal mixed text classification model (MHA-MST). We use the multi head self attention mechanism to focus on the words that have a significant impact on classification, use multi-channel CNN with convolution kernels of different sizes extract local features of texts with different granularity, use stack Ordered Neurons-Long Short-Term Memory(ON-LSTM) extract context semantic information and long-distance dependency, the spatial-temporal mixed features extracted by multi-channel CNN and stacked ON-LSTM are sent to softmax classifier to obtain classification results. The experimental results on Taobao review data set and THUCNews data set show that the proposed MAHMCO model achieves better classification results than other models.