Adaptive Sampling for Coarse Ranking
  We consider the problem of active coarse ranking, where the goal is to sort
items according to their means into clusters of pre-specified sizes, by
adaptively sampling from their reward distributions. This setting is useful in
many social science applications involving human raters and the approximate
rank of every item is desired. Approximate or coarse ranking can significantly
reduce the number of ratings required in comparison to the number needed to
find an exact ranking. We propose a computationally efficient PAC algorithm
LUCBRank for coarse ranking, and derive an upper bound on its sample
complexity. We also derive a nearly matching distribution-dependent lower
bound. Experiments on synthetic as well as real-world data show that LUCBRank
performs better than state-of-the-art baseline methods, even when these methods
have the advantage of knowing the underlying parametric model.
