Learning What Data to Learn
  Machine learning is essentially the sciences of playing with data. An
adaptive data selection strategy, enabling to dynamically choose different data
at various training stages, can reach a more effective model in a more
efficient way. In this paper, we propose a deep reinforcement learning
framework, which we call \emph{\textbf{N}eural \textbf{D}ata \textbf{F}ilter}
(\textbf{NDF}), to explore automatic and adaptive data selection in the
training process. In particular, NDF takes advantage of a deep neural network
to adaptively select and filter important data instances from a sequential
stream of training data, such that the future accumulative reward (e.g., the
convergence speed) is maximized. In contrast to previous studies in data
selection that is mainly based on heuristic strategies, NDF is quite generic
and thus can be widely suitable for many machine learning tasks. Taking neural
network training with stochastic gradient descent (SGD) as an example,
comprehensive experiments with respect to various neural network modeling
(e.g., multi-layer perceptron networks, convolutional neural networks and
recurrent neural networks) and several applications (e.g., image classification
and text understanding) demonstrate that NDF powered SGD can achieve comparable
accuracy with standard SGD process by using less data and fewer iterations.
