Regularized Frank-Wolfe for Dense CRFs: Generalizing Mean Field and
  Beyond
  We introduce regularized Frank-Wolfe, a general and effective algorithm for
inference and learning of dense conditional random fields (CRFs). The algorithm
optimizes a nonconvex continuous relaxation of the CRF inference problem using
vanilla Frank-Wolfe with approximate updates, which are equivalent to
minimizing a regularized energy function. Our proposed method is a
generalization of existing algorithms such as mean field or concave-convex
procedure. This perspective not only offers a unified analysis of these
algorithms, but also allows an easy way of exploring different variants that
potentially yield better performance. We illustrate this in our empirical
results on standard semantic segmentation datasets, where several
instantiations of our regularized Frank-Wolfe outperform mean field inference,
both as a standalone component and as an end-to-end trainable layer in a neural
network. We also show that dense CRFs, coupled with our new algorithms, produce
significant improvements over strong CNN baselines.
