Convex Clustering: Model, Theoretical Guarantee and Efficient Algorithm
  Clustering is a fundamental problem in unsupervised learning. Popular methods
like K-means, may suffer from poor performance as they are prone to get stuck
in its local minima. Recently, the sum-of-norms (SON) model (also known as the
clustering path) has been proposed in Pelckmans et al. (2005), Lindsten et al.
(2011) and Hocking et al. (2011). The perfect recovery properties of the convex
clustering model with uniformly weighted all pairwise-differences
regularization have been proved by Zhu et al. (2014) and Panahi et al. (2017).
However, no theoretical guarantee has been established for the general weighted
convex clustering model, where better empirical results have been observed. In
the numerical optimization aspect, although algorithms like the alternating
direction method of multipliers (ADMM) and the alternating minimization
algorithm (AMA) have been proposed to solve the convex clustering model (Chi
and Lange, 2015), it still remains very challenging to solve large-scale
problems. In this paper, we establish sufficient conditions for the perfect
recovery guarantee of the general weighted convex clustering model, which
include and improve existing theoretical results as special cases. In addition,
we develop a semismooth Newton based augmented Lagrangian method for solving
large-scale convex clustering problems. Extensive numerical experiments on both
simulated and real data demonstrate that our algorithm is highly efficient and
robust for solving large-scale problems. Moreover, the numerical results also
show the superior performance and scalability of our algorithm comparing to the
existing first-order methods. In particular, our algorithm is able to solve a
convex clustering problem with 200,000 points in $\mathbb{R}^3$ in about 6
minutes.
