{'fraction_non_alphanumeric': 0.011778563015312132, 'fraction_numerical': 0.0, 'mean_word_length': 5.390977443609023, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Modelless Data Quality Improvement of Streaming Synchrophasor Measurements by Exploiting the Low-Rank Hankel Structure

This paper presents a new framework to improve the quality of streaming synchrophasor measurements with the existence of missing data and bad data. The method exploits the low-rank property of the Hankel structure to identify and correct bad data, as well as to estimate and fill in the missing data. The method is advantageous compared to existing methods in the literature that only estimate missing data by leveraging the low-rank property of the synchrophasor data observation matrix. The proposed algorithm can efficiently differentiate event data from bad data, even in the existence of simultaneous and consecutive bad data. The algorithm has been verified through numerical experiments on recorded synchrophasor datasets.