Universal Approximation of Markov Kernels by Shallow Stochastic
  Feedforward Networks
  We establish upper bounds for the minimal number of hidden units for which a
binary stochastic feedforward network with sigmoid activation probabilities and
a single hidden layer is a universal approximator of Markov kernels. We show
that each possible probabilistic assignment of the states of $n$ output units,
given the states of $k\geq1$ input units, can be approximated arbitrarily well
by a network with $2^{k-1}(2^{n-1}-1)$ hidden units.
