Exact marginal inference in Latent Dirichlet Allocation
  Assume we have potential "causes" $z\in Z$, which produce "events" $w$ with
known probabilities $\beta(w|z)$. We observe $w_1,w_2,...,w_n$, what can we say
about the distribution of the causes? A Bayesian estimate will assume a prior
on distributions on $Z$ (we assume a Dirichlet prior) and calculate a
posterior. An average over that posterior then gives a distribution on $Z$,
which estimates how much each cause $z$ contributed to our observations. This
is the setting of Latent Dirichlet Allocation, which can be applied e.g. to
topics "producing" words in a document. In this setting usually the number of
observed words is large, but the number of potential topics is small. We are
here interested in applications with many potential "causes" (e.g. locations on
the globe), but only a few observations. We show that the exact Bayesian
estimate can be computed in linear time (and constant space) in $|Z|$ for a
given upper bound on $n$ with a surprisingly simple formula. We generalize this
algorithm to the case of sparse probabilities $\beta(w|z)$, in which we only
need to assume that the tree width of an "interaction graph" on the
observations is limited. On the other hand we also show that without such
limitation the problem is NP-hard.
