Consistent Batch Normalization for Weighted Loss in Imbalanced-Data
  Environment
  In this study, classification problems based on feedforward neural networks
in a data-imbalanced environment are considered. Learning from an imbalanced
dataset is one of the most important practical problems in the field of machine
learning. A weighted loss function (WLF) based on a cost-sensitive approach is
a well-known and effective method for imbalanced datasets. A combination of WLF
and batch normalization (BN) is considered in this study. BN is considered as a
powerful standard technique in the recent developments in deep learning. A
simple combination of both methods leads to a size-inconsistency problem due to
a mismatch between the interpretations of the effective size of the dataset in
both methods. A simple modification to BN, called weighted BN (WBN), is
proposed to correct the size mismatch. The idea of WBN is simple and natural.
The proposed method in a data-imbalanced environment is validated using
numerical experiments.
