Estimating Renyi Entropy of Discrete Distributions
  It was recently shown that estimating the Shannon entropy $H({\rm p})$ of a
discrete $k$-symbol distribution ${\rm p}$ requires $\Theta(k/\log k)$ samples,
a number that grows near-linearly in the support size. In many applications
$H({\rm p})$ can be replaced by the more general R\'enyi entropy of order
$\alpha$, $H_\alpha({\rm p})$. We determine the number of samples needed to
estimate $H_\alpha({\rm p})$ for all $\alpha$, showing that $\alpha < 1$
requires a super-linear, roughly $k^{1/\alpha}$ samples, noninteger $\alpha>1$
requires a near-linear $k$ samples, but, perhaps surprisingly, integer
$\alpha>1$ requires only $\Theta(k^{1-1/\alpha})$ samples. Furthermore,
developing on a recently established connection between polynomial
approximation and estimation of additive functions of the form $\sum_{x} f({\rm
p}_x)$, we reduce the sample complexity for noninteger values of $\alpha$ by a
factor of $\log k$ compared to the empirical estimator. The estimators
achieving these bounds are simple and run in time linear in the number of
samples. Our lower bounds provide explicit constructions of distributions with
different R\'enyi entropies that are hard to distinguish.
