{'fraction_non_alphanumeric': 0.023048327137546468, 'fraction_numerical': 0.0007434944237918215, 'mean_word_length': 5.40952380952381, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A minimum chi‐squared method for indirect parameter estimation from Poisson data

In this paper, we introduce a new algorithm for estimating non‐negative parameters from Poisson observations of a linear transformation of the parameters. The proposed objective function fits both a weighted least squares (WLS) and a minimum χ2 estimation framework, and results in a convex optimization problem. Unlike conventional WLS methods, the weights do not need to be estimated from the datas, but are incorporated in the objective function. The iterative algorithm is derived from an alternating projection procedure in which “distance” is determined by the chi‐squared test statistic, which is interpreted as a measure of the discrepancy between two distributions. This may be viewed as an alternative to the Kullback‐Leibler divergence which corresponds to the maximum likelihood (ML) estimation. The algorithm is similar in form to, and shares many properties with, the expectation maximization algorithm for ML estimation. In particular, we show that every limit point of the algorithm is an estimator, and the sequence of projected (by the linear transformation into the data space) means converge. Despite the similarities, we show that the new estimators are quite distinct from ML estimators, and obtain conditions under which they are identical.