End-to-end Training of CNN-CRF via Differentiable Dual-Decomposition
  Modern computer vision (CV) is often based on convolutional neural networks
(CNNs) that excel at hierarchical feature extraction. The previous generation
of CV approaches was often based on conditional random fields (CRFs) that excel
at modeling flexible higher order interactions. As their benefits are
complementary they are often combined. However, these approaches generally use
mean-field approximations and thus, arguably, did not directly optimize the
real problem. Here we revisit dual-decomposition-based approaches to CRF
optimization, an alternative to the mean-field approximation. These algorithms
can efficiently and exactly solve sub-problems and directly optimize a convex
upper bound of the real problem, providing optimality certificates on the way.
Our approach uses a novel fixed-point iteration algorithm which enjoys
dual-monotonicity, dual-differentiability and high parallelism. The whole
system, CRF and CNN can thus be efficiently trained using back-propagation. We
demonstrate the effectiveness of our system on semantic image segmentation,
showing consistent improvement over baseline models.
