{'fraction_non_alphanumeric': 0.022530329289428077, 'fraction_numerical': 0.0008665511265164644, 'mean_word_length': 5.3812154696132595, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A Contracting BFGS Update in Quasi-Newton Methods for Unconstrained Optimization

Unconstrained optimization problems, arise in many practical applications. Especially, significant improvement in deep learning training came from the Quasi-Newton methods. They exploit the idea of building up curvature information as the iterations of the training method are progressing. BFGS update in Quasi-Newton methods is the most commonly used update rule for training deep neural networks. The accuracy of computed search direction depends largely on how sensitive the Hessian approximation matrix Bk+1 is to small changes. The larger distribution of eigenvalues of the matrix will cause more sensation. In this paper, we propose a contracting BFGS update (C-BFGS), in order to contract the interval of distribution. The new update retains the Hessian approximation matrix positive definiteness, so that it makes sure the search direction down. For a quadratic positive definite function, the search directions generated by C-BFGS under the exact line search are Gconjugate, and the Quasi-Newton method with C-BFGS update satisfies quadratic termination property.