A Bayesian non-parametric method for clustering high-dimensional binary
  data
  In many real life problems, objects are described by large number of binary
features. For instance, documents are characterized by presence or absence of
certain keywords; cancer patients are characterized by presence or absence of
certain mutations etc. In such cases, grouping together similar
objects/profiles based on such high dimensional binary features is desirable,
but challenging. Here, I present a Bayesian non parametric algorithm for
clustering high dimensional binary data. It uses a Dirichlet Process (DP)
mixture model and simulated annealing to not only cluster binary data, but also
find optimal number of clusters in the data. The performance of the algorithm
was evaluated and compared with other algorithms using simulated datasets. It
outperformed all other clustering methods that were tested in the simulation
studies. It was also used to cluster real datasets arising from document
analysis, handwritten image analysis and cancer research. It successfully
divided a set of documents based on their topics, hand written images based on
different styles of writing digits and identified tissue and mutation
specificity of chemotherapy treatments.
