Unified Regularity Measures for Sample-wise Learning and Generalization
  Fundamental machine learning theory shows that different samples contribute
unequally both in learning and testing processes. Contemporary studies on DNN
imply that such sample difference is rooted on the distribution of intrinsic
pattern information, namely sample regularity. Motivated by the recent
discovery on network memorization and generalization, we proposed a pair of
sample regularity measures for both processes with a formulation-consistent
representation. Specifically, cumulative binary training/generalizing loss
(CBTL/CBGL), the cumulative number of correct classiffcations of the
training/testing sample within training stage, is proposed to quantize the
stability in memorization-generalization process; while
forgetting/mal-generalizing events, i.e., the mis-classification of previously
learned or generalized sample, are utilized to represent the uncertainty of
sample regularity with respect to optimization dynamics. Experiments validated
the effectiveness and robustness of the proposed approaches for mini-batch SGD
optimization. Further applications on training/testing sample selection show
the proposed measures sharing the unified computing procedure could benefit for
both tasks.
