Copula Representations and Error Surface Projections for the Exclusive
  Or Problem
  The exclusive or (xor) function is one of the simplest examples that
illustrate why nonlinear feedforward networks are superior to linear regression
for machine learning applications. We review the xor representation and
approximation problems and discuss their solutions in terms of probabilistic
logic and associative copula functions. After briefly reviewing the
specification of feedforward networks, we compare the dynamics of learned error
surfaces with different activation functions such as RELU and tanh through a
set of colorful three-dimensional charts. The copula representations extend xor
from Boolean to real values, thereby providing a convenient way to demonstrate
the concept of cross-validation on in-sample and out-sample data sets. Our
approach is pedagogical and is meant to be a machine learning prolegomenon.
