{'fraction_non_alphanumeric': 0.018261964735516372, 'fraction_numerical': 0.0006297229219143577, 'mean_word_length': 5.20703125, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Pushing the Limits of Sparse Support Recovery Using Correlation Information

A new framework for the problem of sparse support recovery is proposed, which exploits statistical information about the unknown sparse signal in the form of its correlation. A key contribution of this paper is to show that if existing algorithms can recover sparse support of size s, then using such correlation information, the guaranteed size of recoverable support can be increased to O(s2), although the sparse signal itself may not be recoverable. This is proved to be possible by (a) formulating the sparse support recovery problem in terms of the covariance matrix of the measurements, and (b) designing a suitable measurement/sampling matrix which inherently exploits the correlation priors. The so-called Khatri-Rao product of the measurement matrix is shown to play an important role in deciding the level of recoverable sparsity. A systematic analysis of the proposed framework is also presented for the cases when the covariance matrix is only approximately known, by estimating it from finite number of measurements, obtained from the Multiple Measurement Vector (MMV) model. In this case, the use of LASSO on the estimated covariance matrix is proposed for recovering the support. However, the recovery may not be exact and hence a probabilistic guarantee is developed both for sources with arbitrary distribution as well as for Gaussian sources. In the latter case, it is shown that such recovery can happen with overwhelming probability as the number of available measurement vectors increases.