Equivalence of approximation by convolutional neural networks and
  fully-connected networks
  Convolutional neural networks are the most widely used type of neural
networks in applications. In mathematical analysis, however, mostly
fully-connected networks are studied. In this paper, we establish a connection
between both network architectures. Using this connection, we show that all
upper and lower bounds concerning approximation rates of {fully-connected}
neural networks for functions $f \in \mathcal{C}$ -- for an arbitrary function
class $\mathcal{C}$ -- translate to essentially the same bounds concerning
approximation rates of convolutional neural networks for functions $f \in
{\mathcal{C}^{equi}}$, with the class ${\mathcal{C}^{equi}}$ consisting of all
translation equivariant functions whose first coordinate belongs to
$\mathcal{C}$. All presented results consider exclusively the case of
convolutional neural networks without any pooling operation and with circular
convolutions, i.e., not based on zero-padding.
