Shallow Neural Network can Perfectly Classify an Object following
  Separable Probability Distribution
  Guiding the design of neural networks is of great importance to save enormous
resources consumed on empirical decisions of architectural parameters. This
paper constructs shallow sigmoid-type neural networks that achieve 100%
accuracy in classification for datasets following a linear separability
condition. The separability condition in this work is more relaxed than the
widely used linear separability. Moreover, the constructed neural network
guarantees perfect classification for any datasets sampled from a separable
probability distribution. This generalization capability comes from the
saturation of sigmoid function that exploits small margins near the boundaries
of intervals formed by the separable probability distribution.
