The Shapley Value of Classifiers in Ensemble Games
  What is the value of an individual model in an ensemble of binary
classifiers? We answer this question by introducing a class of transferable
utility cooperative games called \textit{ensemble games}. In machine learning
ensembles, pre-trained models cooperate to make classification decisions. To
quantify the importance of models in these ensemble games, we define
\textit{Troupe} -- an efficient algorithm which allocates payoffs based on
approximate Shapley values of the classifiers. We argue that the Shapley value
of models in these games is an effective decision metric for choosing a high
performing subset of models from the ensemble. Our analytical findings prove
that our Shapley value estimation scheme is precise and scalable; its
performance increases with size of the dataset and ensemble. Empirical results
on real world graph classification tasks demonstrate that our algorithm
produces high quality estimates of the Shapley value. We find that Shapley
values can be utilized for ensemble pruning, and that adversarial models
receive a low valuation. Complex classifiers are frequently found to be
responsible for both correct and incorrect classification decisions.
