On Structured Filtering-Clustering: Global Error Bound and Optimal
  First-Order Algorithms
  The filtering-clustering models, including trend filtering and convex
clustering, have become an important source of ideas and modeling tools in
machine learning and related fields. The statistical guarantee of optimal
solutions in these models has been extensively studied yet the investigations
on the computational aspect have remained limited. In particular, practitioners
often employ the first-order algorithms in real-world applications and are
impressed by their superior performance regardless of ill-conditioned
structures of difference operator matrices, thus leaving open the problem of
understanding the convergence property of first-order algorithms. This paper
settles this open problem and contributes to the broad interplay between
statistics and optimization by identifying a \textit{global error bound}
condition, which is satisfied by a large class of dual filtering-clustering
problems, and designing a class of \textit{generalized dual gradient ascent}
algorithm, which is \textit{optimal} first-order algorithms in deterministic,
finite-sum and online settings. Our results are new and help explain why the
filtering-clustering models can be efficiently solved by first-order
algorithms. We also provide the detailed convergence rate analysis for the
proposed algorithms in different settings, shedding light on their potential to
solve the filtering-clustering models efficiently. We also conduct experiments
on real datasets and the numerical results demonstrate the effectiveness of our
algorithms.
