Safe Screening for Multi-Task Feature Learning with Multiple Data
  Matrices
  Multi-task feature learning (MTFL) is a powerful technique in boosting the
predictive performance by learning multiple related
classification/regression/clustering tasks simultaneously. However, solving the
MTFL problem remains challenging when the feature dimension is extremely large.
In this paper, we propose a novel screening rule---that is based on the dual
projection onto convex sets (DPC)---to quickly identify the inactive
features---that have zero coefficients in the solution vectors across all
tasks. One of the appealing features of DPC is that: it is safe in the sense
that the detected inactive features are guaranteed to have zero coefficients in
the solution vectors across all tasks. Thus, by removing the inactive features
from the training phase, we may have substantial savings in the computational
cost and memory usage without sacrificing accuracy. To the best of our
knowledge, it is the first screening rule that is applicable to sparse models
with multiple data matrices. A key challenge in deriving DPC is to solve a
nonconvex problem. We show that we can solve for the global optimum efficiently
via a properly chosen parametrization of the constraint set. Moreover, DPC has
very low computational cost and can be integrated with any existing solvers. We
have evaluated the proposed DPC rule on both synthetic and real data sets. The
experiments indicate that DPC is very effective in identifying the inactive
features---especially for high dimensional data---which leads to a speedup up
to several orders of magnitude.
