Controlled Sparsity Kernel Learning
  Multiple Kernel Learning(MKL) on Support Vector Machines(SVMs) has been a
popular front of research in recent times due to its success in application
problems like Object Categorization. This success is due to the fact that MKL
has the ability to choose from a variety of feature kernels to identify the
optimal kernel combination. But the initial formulation of MKL was only able to
select the best of the features and misses out many other informative kernels
presented. To overcome this, the Lp norm based formulation was proposed by
Kloft et. al. This formulation is capable of choosing a non-sparse set of
kernels through a control parameter p. Unfortunately, the parameter p does not
have a direct meaning to the number of kernels selected. We have observed that
stricter control over the number of kernels selected gives us an edge over
these techniques in terms of accuracy of classification and also helps us to
fine tune the algorithms to the time requirements at hand. In this work, we
propose a Controlled Sparsity Kernel Learning (CSKL) formulation that can
strictly control the number of kernels which we wish to select. The CSKL
formulation introduces a parameter t which directly corresponds to the number
of kernels selected. It is important to note that a search in t space is finite
and fast as compared to p. We have also provided an efficient Reduced Gradient
Descent based algorithm to solve the CSKL formulation, which is proven to
converge. Through our experiments on the Caltech101 Object Categorization
dataset, we have also shown that one can achieve better accuracies than the
previous formulations through the right choice of t.
