Making Pre-trained Language Models Good Long-tailed Learners
  Prompt-tuning has shown appealing performance in few-shot classification by
virtue of its capability in effectively exploiting pre-trained knowledge. This
motivates us to check the hypothesis that prompt-tuning is also a promising
choice for long-tailed classification, since the tail classes are intuitively
few-shot ones. To achieve this aim, we conduct empirical studies to examine the
hypothesis. The results demonstrate that prompt-tuning exactly makes
pre-trained language models at least good long-tailed learners. For intuitions
on why prompt-tuning can achieve good performance in long-tailed
classification, we carry out an in-depth analysis by progressively bridging the
gap between prompt-tuning and commonly used fine-tuning. The summary is that
the classifier structure and parameterization form the key to making good
long-tailed learners, in comparison with the less important input structure.
Finally, we verify the applicability of our finding to few-shot classification.
