The Rich Get Richer: Disparate Impact of Semi-Supervised Learning
  Semi-supervised learning (SSL) has demonstrated its potential to improve the
model accuracy for a variety of learning tasks when the high-quality supervised
data is severely limited. Although it is often established that the average
accuracy for the entire population of data is improved, it is unclear how SSL
fares with different sub-populations. Understanding the above question has
substantial fairness implications when different sub-populations are defined by
the demographic groups that we aim to treat fairly. In this paper, we reveal
the disparate impacts of deploying SSL: the sub-population who has a higher
baseline accuracy without using SSL (the "rich" one) tends to benefit more from
SSL; while the sub-population who suffers from a low baseline accuracy (the
"poor" one) might even observe a performance drop after adding the SSL module.
We theoretically and empirically establish the above observation for a broad
family of SSL algorithms, which either explicitly or implicitly use an
auxiliary "pseudo-label". Experiments on a set of image and text classification
tasks confirm our claims. We introduce a new metric, Benefit Ratio, and promote
the evaluation of the fairness of SSL (Equalized Benefit Ratio). We further
discuss how the disparate impact can be mitigated. We hope our paper will alarm
the potential pitfall of using SSL and encourage a multifaceted evaluation of
future SSL algorithms.
