Prune Responsibly
  Irrespective of the specific definition of fairness in a machine learning
application, pruning the underlying model affects it. We investigate and
document the emergence and exacerbation of undesirable per-class performance
imbalances, across tasks and architectures, for almost one million categories
considered across over 100K image classification models that undergo a pruning
process.We demonstrate the need for transparent reporting, inclusive of bias,
fairness, and inclusion metrics, in real-life engineering decision-making
around neural network pruning. In response to the calls for quantitative
evaluation of AI models to be population-aware, we present neural network
pruning as a tangible application domain where the ways in which
accuracy-efficiency trade-offs disproportionately affect underrepresented or
outlier groups have historically been overlooked. We provide a simple,
Pareto-based framework to insert fairness considerations into value-based
operating point selection processes, and to re-evaluate pruning technique
choices.
