Simon Says: Evaluating and Mitigating Bias in Pruned Neural Networks
  with Knowledge Distillation
  In recent years the ubiquitous deployment of AI has posed great concerns in
regards to algorithmic bias, discrimination, and fairness. Compared to
traditional forms of bias or discrimination caused by humans, algorithmic bias
generated by AI is more abstract and unintuitive therefore more difficult to
explain and mitigate. A clear gap exists in the current literature on
evaluating and mitigating bias in pruned neural networks. In this work, we
strive to tackle the challenging issues of evaluating, mitigating, and
explaining induced bias in pruned neural networks. Our paper makes three
contributions. First, we propose two simple yet effective metrics, Combined
Error Variance (CEV) and Symmetric Distance Error (SDE), to quantitatively
evaluate the induced bias prevention quality of pruned models. Second, we
demonstrate that knowledge distillation can mitigate induced bias in pruned
neural networks, even with unbalanced datasets. Third, we reveal that model
similarity has strong correlations with pruning induced bias, which provides a
powerful method to explain why bias occurs in pruned neural networks. Our code
is available at https://github.com/codestar12/pruning-distilation-bias
