Stretching the Effectiveness of MLE from Accuracy to Bias for Pairwise
  Comparisons
  A number of applications (e.g., AI bot tournaments, sports, peer grading,
crowdsourcing) use pairwise comparison data and the Bradley-Terry-Luce (BTL)
model to evaluate a given collection of items (e.g., bots, teams, students,
search results). Past work has shown that under the BTL model, the widely-used
maximum-likelihood estimator (MLE) is minimax-optimal in estimating the item
parameters, in terms of the mean squared error. However, another important
desideratum for designing estimators is fairness. In this work, we consider
fairness modeled by the notion of bias in statistics. We show that the MLE
incurs a suboptimal rate in terms of bias. We then propose a simple
modification to the MLE, which "stretches" the bounding box of the
maximum-likelihood optimizer by a small constant factor from the underlying
ground truth domain. We show that this simple modification leads to an improved
rate in bias, while maintaining minimax-optimality in the mean squared error.
In this manner, our proposed class of estimators provably improves fairness
represented by bias without loss in accuracy.
