{'fraction_non_alphanumeric': 0.015317286652078774, 'fraction_numerical': 0.005105762217359592, 'mean_word_length': 5.411214953271028, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Difference in Euclidean Norm Can Cause Semantic Divergence in Batch Normalization

In this paper, we show that the difference in Euclidean norm of samples can make a contribution to the semantic divergence and even confusion, after the spatial translation and scaling transformation in batch normalization. To address this issue, we propose an intuitive but effective method to equalize the Euclidean norms of sample vectors. Concretely, we l2-normalize each sample vector before batch normalization, and therefore the sample vectors are of the same magnitude. Since the proposed method combines the l2 normalization and batch normalization, we name our method as L2BN. The L2BN can strengthen the compactness of intraclass features and enlarge the discrepancy of inter-class features. In addition, it can help the gradient converge to a stable scale. TheL2BN is easy to implement and can exert its effect without any additional parameters and hyper-parameters. Therefore, it can be used as a basic normalization method for neural networks. We evaluate the effectiveness ofL2BN through extensive experiments with various models on image classification and acoustic scene classification tasks. The experimental results demonstrate that the L2BN is able to boost the generalization ability of various neural network models and achieve considerable performance improvements.