Inconsistent parameter estimation in Markov random fields: Benefits in
  the computation-limited setting
  Consider the problem of joint parameter estimation and prediction in a Markov
random field: i.e., the model parameters are estimated on the basis of an
initial set of data, and then the fitted model is used to perform prediction
(e.g., smoothing, denoising, interpolation) on a new noisy observation. Working
under the restriction of limited computation, we analyze a joint method in
which the \emph{same convex variational relaxation} is used to construct an
M-estimator for fitting parameters, and to perform approximate marginalization
for the prediction step. The key result of this paper is that in the
computation-limited setting, using an inconsistent parameter estimator (i.e.,
an estimator that returns the ``wrong'' model even in the infinite data limit)
can be provably beneficial, since the resulting errors can partially compensate
for errors made by using an approximate prediction technique. En route to this
result, we analyze the asymptotic properties of M-estimators based on convex
variational relaxations, and establish a Lipschitz stability property that
holds for a broad class of variational methods. We show that joint
estimation/prediction based on the reweighted sum-product algorithm
substantially outperforms a commonly used heuristic based on ordinary
sum-product.
