Partial order: Finding Consensus among Uncertain Feature Attributions
  Post-hoc feature importance is progressively being employed to explain
decisions of complex machine learning models. Yet in practice, reruns of the
training algorithm and/or the explainer can result in contradicting statements
of feature importance, henceforth reducing trust in those techniques. A
possible avenue to address this issue is to develop strategies to aggregate
diverse explanations about feature importance. While the arithmetic mean, which
yields a total order, has been advanced, we introduce an alternative: the
consensus among multiple models, which results in partial orders. The two
aggregation strategies are compared using Integrated Gradients and Shapley
values on two regression datasets, and we show that a large portion of the
information provided by the mean aggregation is not supported by the consensus
of each individual model, raising suspicion on the trustworthiness of this
practice.
