On transversality of bent hyperplane arrangements and the topological
  expressiveness of ReLU neural networks
  Let F:R^n -> R be a feedforward ReLU neural network. It is well-known that
for any choice of parameters, F is continuous and piecewise (affine) linear. We
lay some foundations for a systematic investigation of how the architecture of
F impacts the geometry and topology of its possible decision regions for binary
classification tasks. Following the classical progression for smooth functions
in differential topology, we first define the notion of a generic, transversal
ReLU neural network and show that almost all ReLU networks are generic and
transversal. We then define a partially-oriented linear 1-complex in the domain
of F and identify properties of this complex that yield an obstruction to the
existence of bounded connected components of a decision region. We use this
obstruction to prove that a decision region of a generic, transversal ReLU
network F: R^n -> R with a single hidden layer of dimension (n + 1) can have no
more than one bounded connected component.
