Primal-Dual Active-Set Methods for Isotonic Regression and Trend
  Filtering
  Isotonic regression (IR) is a non-parametric calibration method used in
supervised learning. For performing large-scale IR, we propose a primal-dual
active-set (PDAS) algorithm which, in contrast to the state-of-the-art Pool
Adjacent Violators (PAV) algorithm, can be parallized and is easily
warm-started thus well-suited in the online settings. We prove that, like the
PAV algorithm, our PDAS algorithm for IR is convergent and has a work
complexity of O(n), though our numerical experiments suggest that our PDAS
algorithm is often faster than PAV. In addition, we propose PDAS variants (with
safeguarding to ensure convergence) for solving related trend filtering (TF)
problems, providing the results of experiments to illustrate their
effectiveness.
