{'fraction_non_alphanumeric': 0.013082583810302535, 'fraction_numerical': 0.0, 'mean_word_length': 5.341968911917099, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Algorithms for statistical model selection and robust estimation

Computationally intensive algorithms for model selection and robust regression are considered. Particular emphasis is put on regression trees. The QR decomposition is the main computational tool to solve the linear models. Givens rotations are employed to compute the orthogonal factorizations. A new pipelineparallel strategy is proposed for computing the QR decomposition. Algorithms for computing the best subset regression models are investigated. The algorithms extend previously introduced exhaustive and heuristic strategies, which are aimed at solving large-scale model selection problems. An algorithm is proposed to compute the exact least trimmed squares regression. It can efficiently compute the LTS estimators for a range of coverage values. Thus, the coverage parameter h does not need to be known in advance, and the algorithm can be used to examine the degree of contamination of the data. The LTS algorithm is extended to solve the generalized LTS estimation problem of the GLM and SUR model. The singularity problem of the dispersion matrix is avoided by reformulating the estimation problem as a generalized linear least squares problem.