Mitigating deep double descent by concatenating inputs
  The double descent curve is one of the most intriguing properties of deep
neural networks. It contrasts the classical bias-variance curve with the
behavior of modern neural networks, occurring where the number of samples nears
the number of parameters. In this work, we explore the connection between the
double descent phenomena and the number of samples in the deep neural network
setting. In particular, we propose a construction which augments the existing
dataset by artificially increasing the number of samples. This construction
empirically mitigates the double descent curve in this setting. We reproduce
existing work on deep double descent, and observe a smooth descent into the
overparameterized region for our construction. This occurs both with respect to
the model size, and with respect to the number epochs.
