Sparse K-Means with $\ell_{\infty}/\ell_0$ Penalty for High-Dimensional
  Data Clustering
  Sparse clustering, which aims to find a proper partition of an extremely
high-dimensional data set with redundant noise features, has been attracted
more and more interests in recent years. The existing studies commonly solve
the problem in a framework of maximizing the weighted feature contributions
subject to a $\ell_2/\ell_1$ penalty. Nevertheless, this framework has two
serious drawbacks: One is that the solution of the framework unavoidably
involves a considerable portion of redundant noise features in many situations,
and the other is that the framework neither offers intuitive explanations on
why this framework can select relevant features nor leads to any theoretical
guarantee for feature selection consistency.
  In this article, we attempt to overcome those drawbacks through developing a
new sparse clustering framework which uses a $\ell_{\infty}/\ell_0$ penalty.
First, we introduce new concepts on optimal partitions and noise features for
the high-dimensional data clustering problems, based on which the previously
known framework can be intuitively explained in principle. Then, we apply the
suggested $\ell_{\infty}/\ell_0$ framework to formulate a new sparse k-means
model with the $\ell_{\infty}/\ell_0$ penalty ($\ell_0$-k-means for short). We
propose an efficient iterative algorithm for solving the $\ell_0$-k-means. To
deeply understand the behavior of $\ell_0$-k-means, we prove that the solution
yielded by the $\ell_0$-k-means algorithm has feature selection consistency
whenever the data matrix is generated from a high-dimensional Gaussian mixture
model. Finally, we provide experiments with both synthetic data and the Allen
Developing Mouse Brain Atlas data to support that the proposed $\ell_0$-k-means
exhibits better noise feature detection capacity over the previously known
sparse k-means with the $\ell_2/\ell_1$ penalty ($\ell_1$-k-means for short).
