Understanding Self-Paced Learning under Concave Conjugacy Theory
  By simulating the easy-to-hard learning manners of humans/animals, the
learning regimes called curriculum learning~(CL) and self-paced learning~(SPL)
have been recently investigated and invoked broad interests. However, the
intrinsic mechanism for analyzing why such learning regimes can work has not
been comprehensively investigated. To this issue, this paper proposes a concave
conjugacy theory for looking into the insight of CL/SPL. Specifically, by using
this theory, we prove the equivalence of the SPL regime and a latent concave
objective, which is closely related to the known non-convex regularized penalty
widely used in statistics and machine learning. Beyond the previous theory for
explaining CL/SPL insights, this new theoretical framework on one hand
facilitates two direct approaches for designing new SPL models for certain
tasks, and on the other hand can help conduct the latent objective of
self-paced curriculum learning, which is the advanced version of both CL/SPL
and possess advantages of both learning regimes to a certain extent. This
further facilitates a theoretical understanding for SPCL, instead of only
CL/SPL as conventional. Under this theory, we attempt to attain intrinsic
latent objectives of two curriculum forms, the partial order and group
curriculums, which easily follow the theoretical understanding of the
corresponding SPCL regimes.
