{'fraction_non_alphanumeric': 0.014957264957264958, 'fraction_numerical': 0.002136752136752137, 'mean_word_length': 5.50462962962963, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A Systematic Framework Presenting Impact of Dataset Completeness on Data Driven Approaches: A Transmission Line Fault Classification Study

Fault classification is one of the essential steps to ensure safety and power supply reliability of the transmission lines. This paper focuses on the data driven transmission line fault classification approaches and studies the impact of dataset completeness on the validity of data driven approaches. A systematic framework is proposed to verify this impact. Here three data driven approaches, including SVM, Softmax and BPNN, are considered as examples. First, a small dataset and a full dataset are established, which contain partial and complete characteristics of the fault classification problem, respectively. Afterwards, each dataset is randomly divided into training, validation and testing sets to generate the trained model. Next, an additional testing set is constructed to further validate the fault classification accuracy of the trained model. Since the additional testing set is different from both the small and the full dataset, it provides an objective criterion for the effectiveness of the trained model. Numerical experiments in a 500 kV transmission line system prove that the small dataset may result in overfitting and generalization issues, while the full dataset can fully demonstrate the effectiveness of a certain data driven approach.