Convolutional and Residual Networks Provably Contain Lottery Tickets
  The Lottery Ticket Hypothesis continues to have a profound practical impact
on the quest for small scale deep neural networks that solve modern deep
learning tasks at competitive performance. These lottery tickets are identified
by pruning large randomly initialized neural networks with architectures that
are as diverse as their applications. Yet, theoretical insights that attest
their existence have been mostly focused on deep fully-connected feed forward
networks with ReLU activation functions. We prove that also modern
architectures consisting of convolutional and residual layers that can be
equipped with almost arbitrary activation functions can contain lottery tickets
with high probability.
