Feeling the Bern: Adaptive Estimators for Bernoulli Probabilities of
  Pairwise Comparisons
  We study methods for aggregating pairwise comparison data in order to
estimate outcome probabilities for future comparisons among a collection of n
items. Working within a flexible framework that imposes only a form of strong
stochastic transitivity (SST), we introduce an adaptivity index defined by the
indifference sets of the pairwise comparison probabilities. In addition to
measuring the usual worst-case risk of an estimator, this adaptivity index also
captures the extent to which the estimator adapts to instance-specific
difficulty relative to an oracle estimator. We prove three main results that
involve this adaptivity index and different algorithms. First, we propose a
three-step estimator termed Count-Randomize-Least squares (CRL), and show that
it has adaptivity index upper bounded as $\sqrt{n}$ up to logarithmic factors.
We then show that that conditional on the hardness of planted clique, no
computationally efficient estimator can achieve an adaptivity index smaller
than $\sqrt{n}$. Second, we show that a regularized least squares estimator can
achieve a poly-logarithmic adaptivity index, thereby demonstrating a
$\sqrt{n}$-gap between optimal and computationally achievable adaptivity.
Finally, we prove that the standard least squares estimator, which is known to
be optimally adaptive in several closely related problems, fails to adapt in
the context of estimating pairwise probabilities.
