Sign-Constrained Regularized Loss Minimization
  In practical analysis, domain knowledge about analysis target has often been
accumulated, although, typically, such knowledge has been discarded in the
statistical analysis stage, and the statistical tool has been applied as a
black box. In this paper, we introduce sign constraints that are a handy and
simple representation for non-experts in generic learning problems. We have
developed two new optimization algorithms for the sign-constrained regularized
loss minimization, called the sign-constrained Pegasos (SC-Pega) and the
sign-constrained SDCA (SC-SDCA), by simply inserting the sign correction step
into the original Pegasos and SDCA, respectively. We present theoretical
analyses that guarantee that insertion of the sign correction step does not
degrade the convergence rate for both algorithms. Two applications, where the
sign-constrained learning is effective, are presented. The one is exploitation
of prior information about correlation between explanatory variables and a
target variable. The other is introduction of the sign-constrained to
SVM-Pairwise method. Experimental results demonstrate significant improvement
of generalization performance by introducing sign constraints in both
applications.
