RieszNet and ForestRiesz: Automatic Debiased Machine Learning with
  Neural Nets and Random Forests
  Many causal and policy effects of interest are defined by linear functionals
of high-dimensional or non-parametric regression functions.
$\sqrt{n}$-consistent and asymptotically normal estimation of the object of
interest requires debiasing to reduce the effects of regularization and/or
model selection on the object of interest. Debiasing is typically achieved by
adding a correction term to the plug-in estimator of the functional, which
leads to properties such as semi-parametric efficiency, double robustness, and
Neyman orthogonality. We implement an automatic debiasing procedure based on
automatically learning the Riesz representation of the linear functional using
Neural Nets and Random Forests. Our method only relies on black-box evaluation
oracle access to the linear functional and does not require knowledge of its
analytic form. We propose a multitasking Neural Net debiasing method with
stochastic gradient descent minimization of a combined Riesz representer and
regression loss, while sharing representation layers for the two functions. We
also propose a Random Forest method which learns a locally linear
representation of the Riesz function. Even though our method applies to
arbitrary functionals, we experimentally find that it performs well compared to
the state of art neural net based algorithm of Shi et al. (2019) for the case
of the average treatment effect functional. We also evaluate our method on the
problem of estimating average marginal effects with continuous treatments,
using semi-synthetic data of gasoline price changes on gasoline demand.
