{'fraction_non_alphanumeric': 0.018147086914995225, 'fraction_numerical': 0.0, 'mean_word_length': 5.128654970760234, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Research and Design of Internet Public Opinion Analysis System

Internet is becoming a spreading platform for the public opinion. It is important to grasp the internet public opinion in time and understand the trends of their opinion correctly. Text classification plays a fundamental role in a number of information management and retrieval tasks. But web-page classification is much more difficult than pure-text classification due to a large variety of noisy information embedded in Web pages. In this paper, we propose a system scheme for the analysis of the Internet public opinion (IPO).We apply web-page classification through summarization to extract the most relevant content from the Web pages and then pass them to standard text classification algorithms (NB or SVM). We comprehensive use text classification and text clustering algorithms, which have been shown to be efficient and effective for singly using. Through the result of the experiment,we have proved the superiority of the system system’s architecture in the system design.