{'fraction_non_alphanumeric': 0.02761904761904762, 'fraction_numerical': 0.002857142857142857, 'mean_word_length': 5.07514450867052, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Nonparametric density estimation by a self-consistent neural network

An improvement to the classic adaptive kernel estimator has been made by incorporating first order dynamics in a neural network framework that results in a fully self-consistent probability density function (pdf) estimate. The dynamics give rise to nonlinear interactions between the kernel parameters, resulting in a self-consistent pdf estimate. This is in contrast to the adaptive kernel estimator which is a simple three step procedure. Adaptive kernel estimates have asymptotic convergence rates of O(h/sup 4/) if the errors involved in the pilot estimate can be ignored. This is compared to standard kernel estimators which converge as O(h/sup 2/). By using a fully self-consistent method, this approach is also able to approach the theoretical O(h/sup 4/) convergence rate while providing smoother estimates of the distribution tails than the adaptive kernel estimator. A one-dimensional application to the estimation of a log-normal distribution is included as an example.