Fairness Testing of Deep Image Classification with Adequacy Metrics
  As deep image classification applications, e.g., face recognition, become
increasingly prevalent in our daily lives, their fairness issues raise more and
more concern. It is thus crucial to comprehensively test the fairness of these
applications before deployment. Existing fairness testing methods suffer from
the following limitations: 1) applicability, i.e., they are only applicable for
structured data or text without handling the high-dimensional and abstract
domain sampling in the semantic level for image classification applications; 2)
functionality, i.e., they generate unfair samples without providing testing
criterion to characterize the model's fairness adequacy. To fill the gap, we
propose DeepFAIT, a systematic fairness testing framework specifically designed
for deep image classification applications. DeepFAIT consists of several
important components enabling effective fairness testing of deep image
classification applications: 1) a neuron selection strategy to identify the
fairness-related neurons; 2) a set of multi-granularity adequacy metrics to
evaluate the model's fairness; 3) a test selection algorithm for fixing the
fairness issues efficiently. We have conducted experiments on widely adopted
large-scale face recognition applications, i.e., VGGFace and FairFace. The
experimental results confirm that our approach can effectively identify the
fairness-related neurons, characterize the model's fairness, and select the
most valuable test cases to mitigate the model's fairness issues.
