Quantitative approximation results for complex-valued neural networks
  Until recently, applications of neural networks in machine learning have
almost exclusively relied on real-valued networks. It was recently observed,
however, that complex-valued neural networks (CVNNs) exhibit superior
performance in applications in which the input is naturally complex-valued,
such as MRI fingerprinting. While the mathematical theory of real-valued
networks has, by now, reached some level of maturity, this is far from true for
complex-valued networks. In this paper, we analyze the expressivity of
complex-valued networks by providing explicit quantitative error bounds for
approximating $C^n$ functions on compact subsets of $\mathbb{C}^d$ by
complex-valued neural networks that employ the modReLU activation function,
given by $\sigma(z) = \mathrm{ReLU}(|z| - 1) \, \mathrm{sgn} (z)$, which is one
of the most popular complex activation functions used in practice. We show that
the derived approximation rates are optimal (up to log factors) in the class of
modReLU networks with weights of moderate growth.
