{'fraction_non_alphanumeric': 0.014102564102564103, 'fraction_numerical': 0.00641025641025641, 'mean_word_length': 5.198412698412699, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Computing Maximum Likelihood Estimators of Convex Density Functions

We consider the problem of estimating a density function that is known in advance to be convex. The maximum likelihood estimator is then the solution of a linearly constrained convex minimization problem. This problem turns out to be numerically difficult. We show that interior point algorithms perform well on this class of optimization problem, though for large samples numerical difficulties are still encountered. To eliminate those difficulties, we propose a clustering scheme that is reasonable from a statistical point of view. We display results for problems with up to 40000 observations. We also give a typical picture of the estimated density: a piecewise linear function, with only very few pieces.