An Exact Poly-Time Membership-Queries Algorithm for Extraction a
  three-Layer ReLU Network
  We consider the natural problem of learning a ReLU network from queries,
which was recently re-motivated by model extraction attacks. In this work we
present a polynomial-time algorithm, that under mild general position
assumptions, can learn a depth two ReLU network from queries. We also present a
polynomial-time algorithm, that under mild general position assumptions, can
learn a rich class of depth three ReLU network from queries. For instance, it
can learn most networks in which the number of first layer neurons is smaller
than the dimension and the number of second layer neurons.
  These two results substantially improve the state of the art: Until our work,
polynomial-time algorithms were only shown to learn from queries depth two
networks under the assumption that either the underlying distribution is
Gaussian or that the weights matrix rows are linearly independent. For depth
three or more, there were no known poly-time results.
