A Distributional Framework for Data Valuation
  Shapley value is a classic notion from game theory, historically used to
quantify the contributions of individuals within groups, and more recently
applied to assign values to data points when training machine learning models.
Despite its foundational role, a key limitation of the data Shapley framework
is that it only provides valuations for points within a fixed data set. It does
not account for statistical aspects of the data and does not give a way to
reason about points outside the data set. To address these limitations, we
propose a novel framework -- distributional Shapley -- where the value of a
point is defined in the context of an underlying data distribution. We prove
that distributional Shapley has several desirable statistical properties; for
example, the values are stable under perturbations to the data points
themselves and to the underlying data distribution. We leverage these
properties to develop a new algorithm for estimating values from data, which
comes with formal guarantees and runs two orders of magnitude faster than
state-of-the-art algorithms for computing the (non-distributional) data Shapley
values. We apply distributional Shapley to diverse data sets and demonstrate
its utility in a data market setting.
