The Sum-Product Theorem: A Foundation for Learning Tractable Models
  Inference in expressive probabilistic models is generally intractable, which
makes them difficult to learn and limits their applicability. Sum-product
networks are a class of deep models where, surprisingly, inference remains
tractable even when an arbitrary number of hidden layers are present. In this
paper, we generalize this result to a much broader set of learning problems:
all those where inference consists of summing a function over a semiring. This
includes satisfiability, constraint satisfaction, optimization, integration,
and others. In any semiring, for summation to be tractable it suffices that the
factors of every product have disjoint scopes. This unifies and extends many
previous results in the literature. Enforcing this condition at learning time
thus ensures that the learned models are tractable. We illustrate the power and
generality of this approach by applying it to a new type of structured
prediction problem: learning a nonconvex function that can be globally
optimized in polynomial time. We show empirically that this greatly outperforms
the standard approach of learning without regard to the cost of optimization.
