Joint Probability Estimation Using Tensor Decomposition and Dictionaries
  In this work, we study non-parametric estimation of joint probabilities of a
given set of discrete and continuous random variables from their (empirically
estimated) 2D marginals, under the assumption that the joint probability could
be decomposed and approximated by a mixture of product densities/mass
functions. The problem of estimating the joint probability density function
(PDF) using semi-parametric techniques such as Gaussian Mixture Models (GMMs)
is widely studied. However such techniques yield poor results when the
underlying densities are mixtures of various other families of distributions
such as Laplacian or generalized Gaussian, uniform, Cauchy, etc. Further, GMMs
are not the best choice to estimate joint distributions which are hybrid in
nature, i.e., some random variables are discrete while others are continuous.
We present a novel approach for estimating the PDF using ideas from dictionary
representations in signal processing coupled with low rank tensor
decompositions. To the best our knowledge, this is the first work on estimating
joint PDFs employing dictionaries alongside tensor decompositions. We create a
dictionary of various families of distributions by inspecting the data, and use
it to approximate each decomposed factor of the product in the mixture. Our
approach can naturally handle hybrid $N$-dimensional distributions. We test our
approach on a variety of synthetic and real datasets to demonstrate its
effectiveness in terms of better classification rates and lower error rates,
when compared to state of the art estimators.
