c-Eval: A Unified Metric to Evaluate Feature-based Explanations via
  Perturbation
  In many modern image-classification applications, understanding the cause of
model's prediction can be as critical as the prediction's accuracy itself.
Various feature-based local explanations generation methods have been designed
to give us more insights on the decision of complex classifiers. Nevertheless,
there is no consensus on evaluating the quality of different explanations. In
response to this lack of comprehensive evaluation, we introduce the c-Eval
metric and its corresponding framework to quantify the feature-based local
explanation's quality. Given a classifier's prediction and the corresponding
explanation on that prediction, c-Eval is the minimum-distortion perturbation
that successfully alters the prediction while keeping the explanation's
features unchanged. We then demonstrate how c-Eval can be computed using some
modifications on existing adversarial generation libraries. To show that c-Eval
captures the importance of input's features, we establish the connection
between c-Eval and the features returned by explainers in affine and
nearly-affine classifiers. We then introduce the c-Eval plot, which not only
displays a strong connection between c-Eval and explainers' quality, but also
helps automatically determine explainer's parameters. Since the generation of
c-Eval relies on adversarial generation, we provide a demo of c-Eval on
adversarial-robust models and show that the metric is applicable in those
models. Finally, extensive experiments of explainers on different datasets are
conducted to support the adoption of c-Eval in evaluating explainers'
performance.
