Dynamic Thresholding for Online Distributed Data Selection
  The blessing of ubiquitous data also comes with a curse: the communication,
storage, and labeling of massive, mostly redundant datasets. We seek to solve
this problem at its core, collecting only valuable data and throwing out the
rest via submodular maximization. Specifically, we develop algorithms for the
online and distributed version of the problem, where data selection occurs in
an uncoordinated fashion across multiple data streams. We design a general and
flexible core selection routine for our algorithms which, given any stream of
data, any assessment of its value, and any formulation of its selection cost,
extracts the most valuable subset of the stream up to a constant factor while
using minimal memory. Notably, our methods have the same theoretical guarantees
as their offline counterparts, and, as far as we know, provide the first
guarantees for online distributed submodular optimization in the literature.
Finally, in learning tasks on ImageNet and MNIST, we show that our selection
methods outperform random selection by $5-20\%$.
