Information-Theoretic Bounds and Approximations in Neural Population
  Coding
  While Shannon's mutual information has widespread applications in many
disciplines, for practical applications it is often difficult to calculate its
value accurately for high-dimensional variables because of the curse of
dimensionality. This paper is focused on effective approximation methods for
evaluating mutual information in the context of neural population coding. For
large but finite neural populations, we derive several information-theoretic
asymptotic bounds and approximation formulas that remain valid in
high-dimensional spaces. We prove that optimizing the population density
distribution based on these approximation formulas is a convex optimization
problem which allows efficient numerical solutions. Numerical simulation
results confirmed that our asymptotic formulas were highly accurate for
approximating mutual information for large neural populations. In special
cases, the approximation formulas are exactly equal to the true mutual
information. We also discuss techniques of variable transformation and
dimensionality reduction to facilitate computation of the approximations.
