Self-Organised Factorial Encoding of a Toroidal Manifold
  It is shown analytically how a neural network can be used optimally to encode
input data that is derived from a toroidal manifold. The case of a 2-layer
network is considered, where the output is assumed to be a set of discrete
neural firing events. The network objective function measures the average
Euclidean error that occurs when the network attempts to reconstruct its input
from its output. This optimisation problem is solved analytically for a
toroidal input manifold, and two types of solution are obtained: a joint
encoder in which the network acts as a soft vector quantiser, and a factorial
encoder in which the network acts as a pair of soft vector quantisers (one for
each of the circular subspaces of the torus). The factorial encoder is favoured
for small network sizes when the number of observed firing events is large.
Such self-organised factorial encoding may be used to restrict the size of
network that is required to perform a given encoding task, and will decompose
an input manifold into its constituent submanifolds.
