Analyzing Upper Bounds on Mean Absolute Errors for Deep Neural Network
  Based Vector-to-Vector Regression
  In this paper, we show that, in vector-to-vector regression utilizing deep
neural networks (DNNs), a generalized loss of mean absolute error (MAE) between
the predicted and expected feature vectors is upper bounded by the sum of an
approximation error, an estimation error, and an optimization error. Leveraging
upon error decomposition techniques in statistical learning theory and
non-convex optimization theory, we derive upper bounds for each of the three
aforementioned errors and impose necessary constraints on DNN models. Moreover,
we assess our theoretical results through a set of image de-noising and speech
enhancement experiments. Our proposed upper bounds of MAE for DNN based
vector-to-vector regression are corroborated by the experimental results and
the upper bounds are valid with and without the "over-parametrization"
technique.
