Detect, Replace, Refine: Deep Structured Prediction For Pixel Wise
  Labeling
  Pixel wise image labeling is an interesting and challenging problem with
great significance in the computer vision community. In order for a dense
labeling algorithm to be able to achieve accurate and precise results, it has
to consider the dependencies that exist in the joint space of both the input
and the output variables. An implicit approach for modeling those dependencies
is by training a deep neural network that, given as input an initial estimate
of the output labels and the input image, it will be able to predict a new
refined estimate for the labels. In this context, our work is concerned with
what is the optimal architecture for performing the label improvement task. We
argue that the prior approaches of either directly predicting new label
estimates or predicting residual corrections w.r.t. the initial labels with
feed-forward deep network architectures are sub-optimal. Instead, we propose a
generic architecture that decomposes the label improvement task to three steps:
1) detecting the initial label estimates that are incorrect, 2) replacing the
incorrect labels with new ones, and finally 3) refining the renewed labels by
predicting residual corrections w.r.t. them. Furthermore, we explore and
compare various other alternative architectures that consist of the
aforementioned Detection, Replace, and Refine components. We extensively
evaluate the examined architectures in the challenging task of dense disparity
estimation (stereo matching) and we report both quantitative and qualitative
results on three different datasets. Finally, our dense disparity estimation
network that implements the proposed generic architecture, achieves
state-of-the-art results in the KITTI 2015 test surpassing prior approaches by
a significant margin.
