{'fraction_non_alphanumeric': 0.017543859649122806, 'fraction_numerical': 0.0, 'mean_word_length': 5.541666666666667, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Neural networks : low-memory training and regularization techniques

With the intention of using easy-to-compute and differentiable surrogate models for the optimization of computationally expensive objective functions, we have studied the artificial neural network methodology. We describe in this paper a low-memory Levenberg-Marquardt algorithm for the supervised training of artificial neural networks. It is based on the use of the forward and reverse modes of the algorithmic differentiation. We explain also how we have combined three different strategies of regularization to build neural networks that generalize well.