{'fraction_non_alphanumeric': 0.021293375394321766, 'fraction_numerical': 0.0, 'mean_word_length': 5.75, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Statistical Hypothesis Testing and Variance Analysis for Radio Frequency Interference Identification in Solar Data

This work presents an effective algorithm for radio frequency interference (RFI) identification using dynamic power spectrum statistics in the frequency domain. Statistical signal processing techniques such as hypothesis testing and variance analysis are utilized to derive a test statistic for effective and efficient RFI identification. Starting from the generalized likelihood ratio test (GLRT), we formulate the problem systematically and propose a practical test statistic T(x; f), shown to be F distributed, for RFI identification. A threshold approach working on this test statistic is developed to identify the presence of narrowband RFI in the power spectrum with additive Gaussian noise and/or solar flare background, corresponding to a desired constant false alarm rate (CFAR). Detailed analysis on detector performance and effect of RFI duty cycle are also provided. The proposed statistical test is applied to experimental solar data collected by our frequency-agile solar radio telescope (FASR) subsystem testbed (FST) to demonstrate the robustness and scalability of the algorithm, as well as its capability for real-time implementation.