Rank-smoothed Pairwise Learning In Perceptual Quality Assessment
  Conducting pairwise comparisons is a widely used approach in curating human
perceptual preference data. Typically raters are instructed to make their
choices according to a specific set of rules that address certain dimensions of
image quality and aesthetics. The outcome of this process is a dataset of
sampled image pairs with their associated empirical preference probabilities.
Training a model on these pairwise preferences is a common deep learning
approach. However, optimizing by gradient descent through mini-batch learning
means that the "global" ranking of the images is not explicitly taken into
account. In other words, each step of the gradient descent relies only on a
limited number of pairwise comparisons. In this work, we demonstrate that
regularizing the pairwise empirical probabilities with aggregated rankwise
probabilities leads to a more reliable training loss. We show that training a
deep image quality assessment model with our rank-smoothed loss consistently
improves the accuracy of predicting human preferences.
