Fast Parallel Algorithms for Statistical Subset Selection Problems
  In this paper, we propose a new framework for designing fast parallel
algorithms for fundamental statistical subset selection tasks that include
feature selection and experimental design. Such tasks are known to be weakly
submodular and are amenable to optimization via the standard greedy algorithm.
Despite its desirable approximation guarantees, the greedy algorithm is
inherently sequential and in the worst case, its parallel runtime is linear in
the size of the data. Recently, there has been a surge of interest in a
parallel optimization technique called adaptive sampling which produces
solutions with desirable approximation guarantees for submodular maximization
in exponentially faster parallel runtime. Unfortunately, we show that for
general weakly submodular functions such accelerations are impossible. The
major contribution in this paper is a novel relaxation of submodularity which
we call differential submodularity. We first prove that differential
submodularity characterizes objectives like feature selection and experimental
design. We then design an adaptive sampling algorithm for differentially
submodular functions whose parallel runtime is logarithmic in the size of the
data and achieves strong approximation guarantees. Through experiments, we show
the algorithm's performance is competitive with state-of-the-art methods and
obtains dramatic speedups for feature selection and experimental design
problems.
