Adaptive Approximation and Generalization of Deep Neural Network with
  Intrinsic Dimensionality
  In this study, we prove that an intrinsic low dimensionality of covariates is
the main factor that determines the performance of deep neural networks (DNNs).
DNNs generally provide outstanding empirical performance. Hence, numerous
studies have actively investigated the theoretical properties of DNNs to
understand their underlying mechanisms. In particular, the behavior of DNNs in
terms of high-dimensional data is one of the most critical questions. However,
this issue has not been sufficiently investigated from the aspect of
covariates, although high-dimensional data have practically low intrinsic
dimensionality. In this study, we derive bounds for an approximation error and
a generalization error regarding DNNs with intrinsically low dimensional
covariates. We apply the notion of the Minkowski dimension and develop a novel
proof technique. Consequently, we show that convergence rates of the errors by
DNNs do not depend on the nominal high dimensionality of data, but on its lower
intrinsic dimension. We further prove that the rate is optimal in the minimax
sense. We identify an advantage of DNNs by showing that DNNs can handle a
broader class of intrinsic low dimensional data than other adaptive estimators.
Finally, we conduct a numerical simulation to validate the theoretical results.
