Deep Neural Networks for Estimation and Inference
  We study deep neural networks and their use in semiparametric inference. We
establish novel rates of convergence for deep feedforward neural nets. Our new
rates are sufficiently fast (in some cases minimax optimal) to allow us to
establish valid second-step inference after first-step estimation with deep
learning, a result also new to the literature. Our estimation rates and
semiparametric inference results handle the current standard architecture:
fully connected feedforward neural networks (multi-layer perceptrons), with the
now-common rectified linear unit activation function and a depth explicitly
diverging with the sample size. We discuss other architectures as well,
including fixed-width, very deep networks. We establish nonasymptotic bounds
for these deep nets for a general class of nonparametric regression-type loss
functions, which includes as special cases least squares, logistic regression,
and other generalized linear models. We then apply our theory to develop
semiparametric inference, focusing on causal parameters for concreteness, such
as treatment effects, expected welfare, and decomposition effects. Inference in
many other semiparametric contexts can be readily obtained. We demonstrate the
effectiveness of deep learning with a Monte Carlo analysis and an empirical
application to direct mail marketing.
