BanditPAM: Almost Linear Time $k$-Medoids Clustering via Multi-Armed
  Bandits
  Clustering is a ubiquitous task in data science. Compared to the commonly
used $k$-means clustering, $k$-medoids clustering requires the cluster centers
to be actual data points and support arbitrary distance metrics, which permits
greater interpretability and the clustering of structured objects. Current
state-of-the-art $k$-medoids clustering algorithms, such as Partitioning Around
Medoids (PAM), are iterative and are quadratic in the dataset size $n$ for each
iteration, being prohibitively expensive for large datasets. We propose
BanditPAM, a randomized algorithm inspired by techniques from multi-armed
bandits, that reduces the complexity of each PAM iteration from $O(n^2)$ to
$O(n \log n)$ and returns the same results with high probability, under
assumptions on the data that often hold in practice. As such, BanditPAM matches
state-of-the-art clustering loss while reaching solutions much faster. We
empirically validate our results on several large real-world datasets,
including a coding exercise submissions dataset, the 10x Genomics 68k PBMC
single-cell RNA sequencing dataset, and the MNIST handwritten digits dataset.
In these experiments, we observe that BanditPAM returns the same results as
state-of-the-art PAM-like algorithms up to 4x faster while performing up to
200x fewer distance computations. The improvements demonstrated by BanditPAM
enable $k$-medoids clustering on a wide range of applications, including
identifying cell types in large-scale single-cell data and providing scalable
feedback for students learning computer science online. We also release highly
optimized Python and C++ implementations of our algorithm.
