Differentiable Segmentation of Sequences
  Segmented models are widely used to describe non-stationary sequential data
with discrete change points. Their estimation usually requires solving a mixed
discrete-continuous optimization problem, where the segmentation is the
discrete part and all other model parameters are continuous. A number of
estimation algorithms have been developed that are highly specialized for their
specific model assumptions. The dependence on non-standard algorithms makes it
hard to integrate segmented models in state-of-the-art deep learning
architectures that critically depend on gradient-based optimization techniques.
In this work, we formulate a relaxed variant of segmented models that enables
joint estimation of all model parameters, including the segmentation, with
gradient descent. We build on recent advances in learning continuous warping
functions and propose a novel family of warping functions based on the
two-sided power (TSP) distribution. TSP-based warping functions are
differentiable, have simple closed-form expressions, and can represent
segmentation functions exactly. Our formulation includes the important class of
segmented generalized linear models as a special case, which makes it highly
versatile. We use our approach to model the spread of COVID-19 with Poisson
regression, apply it on a change point detection task, and learn classification
models with concept drift. The experiments show that our approach effectively
learns all these tasks with standard algorithms for gradient descent.
