On the Latent Variable Interpretation in Sum-Product Networks
  One of the central themes in Sum-Product networks (SPNs) is the
interpretation of sum nodes as marginalized latent variables (LVs). This
interpretation yields an increased syntactic or semantic structure, allows the
application of the EM algorithm and to efficiently perform MPE inference. In
literature, the LV interpretation was justified by explicitly introducing the
indicator variables corresponding to the LVs' states. However, as pointed out
in this paper, this approach is in conflict with the completeness condition in
SPNs and does not fully specify the probabilistic model. We propose a remedy
for this problem by modifying the original approach for introducing the LVs,
which we call SPN augmentation. We discuss conditional independencies in
augmented SPNs, formally establish the probabilistic interpretation of the
sum-weights and give an interpretation of augmented SPNs as Bayesian networks.
Based on these results, we find a sound derivation of the EM algorithm for
SPNs. Furthermore, the Viterbi-style algorithm for MPE proposed in literature
was never proven to be correct. We show that this is indeed a correct
algorithm, when applied to selective SPNs, and in particular when applied to
augmented SPNs. Our theoretical results are confirmed in experiments on
synthetic data and 103 real-world datasets.
