Tensorial Mixture Models
  Casting neural networks in generative frameworks is a highly sought-after
endeavor these days. Contemporary methods, such as Generative Adversarial
Networks, capture some of the generative capabilities, but not all. In
particular, they lack the ability of tractable marginalization, and thus are
not suitable for many tasks. Other methods, based on arithmetic circuits and
sum-product networks, do allow tractable marginalization, but their performance
is challenged by the need to learn the structure of a circuit. Building on the
tractability of arithmetic circuits, we leverage concepts from tensor analysis,
and derive a family of generative models we call Tensorial Mixture Models
(TMMs). TMMs assume a simple convolutional network structure, and in addition,
lend themselves to theoretical analyses that allow comprehensive understanding
of the relation between their structure and their expressive properties. We
thus obtain a generative model that is tractable on one hand, and on the other
hand, allows effective representation of rich distributions in an easily
controlled manner. These two capabilities are brought together in the task of
classification under missing data, where TMMs deliver state of the art
accuracies with seamless implementation and design.
