Nonlinear Monte Carlo Method for Imbalanced Data Learning
  For basic machine learning problems, expected error is used to evaluate model
performance. Since the distribution of data is usually unknown, we can make
simple hypothesis that the data are sampled independently and identically
distributed (i.i.d.) and the mean value of loss function is used as the
empirical risk by Law of Large Numbers (LLN). This is known as the Monte Carlo
method. However, when LLN is not applicable, such as imbalanced data problems,
empirical risk will cause overfitting and might decrease robustness and
generalization ability. Inspired by the framework of nonlinear expectation
theory, we substitute the mean value of loss function with the maximum value of
subgroup mean loss. We call it nonlinear Monte Carlo method. In order to use
numerical method of optimization, we linearize and smooth the functional of
maximum empirical risk and get the descent direction via quadratic programming.
With the proposed method, we achieve better performance than SOTA backbone
models with less training steps, and more robustness for basic regression and
imbalanced classification tasks.
