Estimating Divergences in High Dimensions
  The problem of estimating the divergence between 2 high dimensional
distributions with limited samples is an important problem in various fields
such as machine learning. Although previous methods perform well with moderate
dimensional data, their accuracy starts to degrade in situations with 100s of
binary variables. Therefore, we propose the use of decomposable models for
estimating divergences in high dimensional data. These allow us to factorize
the estimated density of the high-dimensional distribution into a product of
lower dimensional functions. We conduct formal and experimental analyses to
explore the properties of using decomposable models in the context of
divergence estimation. To this end, we show empirically that estimating the
Kullback-Leibler divergence using decomposable models from a maximum likelihood
estimator outperforms existing methods for divergence estimation in situations
where dimensionality is high and useful decomposable models can be learnt from
the available data.
