Data Valuation using Reinforcement Learning
  Quantifying the value of data is a fundamental problem in machine learning.
Data valuation has multiple important use cases: (1) building insights about
the learning task, (2) domain adaptation, (3) corrupted sample discovery, and
(4) robust learning. To adaptively learn data values jointly with the target
task predictor model, we propose a meta learning framework which we name Data
Valuation using Reinforcement Learning (DVRL). We employ a data value estimator
(modeled by a deep neural network) to learn how likely each datum is used in
training of the predictor model. We train the data value estimator using a
reinforcement signal of the reward obtained on a small validation set that
reflects performance on the target task. We demonstrate that DVRL yields
superior data value estimates compared to alternative methods across different
types of datasets and in a diverse set of application scenarios. The corrupted
sample discovery performance of DVRL is close to optimal in many regimes (i.e.
as if the noisy samples were known apriori), and for domain adaptation and
robust learning DVRL significantly outperforms state-of-the-art by 14.6% and
10.8%, respectively.
