PIANO: A Fast Parallel Iterative Algorithm for Multinomial and Sparse
  Multinomial Logistic Regression
  Multinomial Logistic Regression is a well-studied tool for classification and
has been widely used in fields like image processing, computer vision and,
bioinformatics, to name a few. Under a supervised classification scenario, a
Multinomial Logistic Regression model learns a weight vector to differentiate
between any two classes by optimizing over the likelihood objective. With the
advent of big data, the inundation of data has resulted in large dimensional
weight vector and has also given rise to a huge number of classes, which makes
the classical methods applicable for model estimation not computationally
viable. To handle this issue, we here propose a parallel iterative algorithm:
Parallel Iterative Algorithm for MultiNomial LOgistic Regression (PIANO) which
is based on the Majorization Minimization procedure, and can parallely update
each element of the weight vectors. Further, we also show that PIANO can be
easily extended to solve the Sparse Multinomial Logistic Regression problem -
an extensively studied problem because of its attractive feature selection
property. In particular, we work out the extension of PIANO to solve the Sparse
Multinomial Logistic Regression problem with l1 and l0 regularizations. We also
prove that PIANO converges to a stationary point of the Multinomial and the
Sparse Multinomial Logistic Regression problems. Simulations were conducted to
compare PIANO with the existing methods, and it was found that the proposed
algorithm performs better than the existing methods in terms of speed of
convergence.
