Limitations of Neural Collapse for Understanding Generalization in Deep
  Learning
  The recent work of Papyan, Han, & Donoho (2020) presented an intriguing
"Neural Collapse" phenomenon, showing a structural property of interpolating
classifiers in the late stage of training. This opened a rich area of
exploration studying this phenomenon. Our motivation is to study the upper
limits of this research program: How far will understanding Neural Collapse
take us in understanding deep learning? First, we investigate its role in
generalization. We refine the Neural Collapse conjecture into two separate
conjectures: collapse on the train set (an optimization property) and collapse
on the test distribution (a generalization property). We find that while Neural
Collapse often occurs on the train set, it does not occur on the test set. We
thus conclude that Neural Collapse is primarily an optimization phenomenon,
with as-yet-unclear connections to generalization. Second, we investigate the
role of Neural Collapse in feature learning. We show simple, realistic
experiments where training longer leads to worse last-layer features, as
measured by transfer-performance on a downstream task. This suggests that
neural collapse is not always desirable for representation learning, as
previously claimed. Finally, we give preliminary evidence of a "cascading
collapse" phenomenon, wherein some form of Neural Collapse occurs not only for
the last layer, but in earlier layers as well. We hope our work encourages the
community to continue the rich line of Neural Collapse research, while also
considering its inherent limitations.
