Penalized k-means algorithms for finding the correct number of clusters
  in a dataset
  In many applications we want to find the number of clusters in a dataset. A
common approach is to use the penalized k-means algorithm with an additive
penalty term linear in the number of clusters. An open problem is estimating
the value of the coefficient of the penalty term. Since estimating the value of
the coefficient in a principled manner appears to be intractable for general
clusters, we investigate "ideal clusters", i.e. identical spherical clusters
with no overlaps and no outlier background noise. In this paper: (a) We derive,
for the case of ideal clusters, rigorous bounds for the coefficient of the
additive penalty. Unsurprisingly, the bounds depend on the correct number of
clusters, which we want to find in the first place. We further show that
additive penalty, even for this simplest case of ideal clusters, typically
produces a weak and often ambiguous signature for the correct number of
clusters. (b) As an alternative, we examine the k-means with multiplicative
penalty, and show that this parameter-free formulation has a stronger, and less
often ambiguous, signature for the correct number of clusters. We also
empirically investigate certain types of deviations from ideal cluster
assumption and show that combination of k-means with additive and
multiplicative penalties can resolve ambiguous solutions.
