Functionally Modular and Interpretable Temporal Filtering for Robust
  Segmentation
  The performance of autonomous systems heavily relies on their ability to
generate a robust representation of the environment. Deep neural networks have
greatly improved vision-based perception systems but still fail in challenging
situations, e.g. sensor outages or heavy weather. These failures are often
introduced by data-inherent perturbations, which significantly reduce the
information provided to the perception system. We propose a functionally
modularized temporal filter, which stabilizes an abstract feature
representation of a single-frame segmentation model using information of
previous time steps. Our filter module splits the filter task into multiple
less complex and more interpretable subtasks. The basic structure of the filter
is inspired by a Bayes estimator consisting of a prediction and an update step.
To make the prediction more transparent, we implement it using a geometric
projection and estimate its parameters. This additionally enables the
decomposition of the filter task into static representation filtering and
low-dimensional motion filtering. Our model can cope with missing frames and is
trainable in an end-to-end fashion. Using photorealistic, synthetic video data,
we show the ability of the proposed architecture to overcome data-inherent
perturbations. The experiments especially highlight advantages introduced by an
interpretable and explicit filter module.
