Asymptotics of Ridge Regression in Convolutional Models
  Understanding generalization and estimation error of estimators for simple
models such as linear and generalized linear models has attracted a lot of
attention recently. This is in part due to an interesting observation made in
machine learning community that highly over-parameterized neural networks
achieve zero training error, and yet they are able to generalize well over the
test samples. This phenomenon is captured by the so called double descent
curve, where the generalization error starts decreasing again after the
interpolation threshold. A series of recent works tried to explain such
phenomenon for simple models. In this work, we analyze the asymptotics of
estimation error in ridge estimators for convolutional linear models. These
convolutional inverse problems, also known as deconvolution, naturally arise in
different fields such as seismology, imaging, and acoustics among others. Our
results hold for a large class of input distributions that include i.i.d.
features as a special case. We derive exact formulae for estimation error of
ridge estimators that hold in a certain high-dimensional regime. We show the
double descent phenomenon in our experiments for convolutional models and show
that our theoretical results match the experiments.
