Layer-Parallel Training with GPU Concurrency of Deep Residual Neural
  Networks via Nonlinear Multigrid
  A Multigrid Full Approximation Storage algorithm for solving Deep Residual
Networks is developed to enable neural network parallelized layer-wise training
and concurrent computational kernel execution on GPUs. This work demonstrates a
10.2x speedup over traditional layer-wise model parallelism techniques using
the same number of compute units.
