Gradient Descent can Learn Less Over-parameterized Two-layer Neural
  Networks on Classification Problems
  Recently, several studies have proven the global convergence and
generalization abilities of the gradient descent method for two-layer ReLU
networks. Most studies especially focused on the regression problems with the
squared loss function, except for a few, and the importance of the positivity
of the neural tangent kernel has been pointed out. On the other hand, the
performance of gradient descent on classification problems using the logistic
loss function has not been well studied, and further investigation of this
problem structure is possible. In this work, we demonstrate that the
separability assumption using a neural tangent model is more reasonable than
the positivity condition of the neural tangent kernel and provide a refined
convergence analysis of the gradient descent for two-layer networks with smooth
activations. A remarkable point of our result is that our convergence and
generalization bounds have much better dependence on the network width in
comparison to related studies. Consequently, our theory provides a
generalization guarantee for less over-parameterized two-layer networks, while
most studies require much higher over-parameterization.
