Understanding Negative Samples in Instance Discriminative
  Self-supervised Representation Learning
  Instance discriminative self-supervised representation learning has been
attracted attention thanks to its unsupervised nature and informative feature
representation for downstream tasks. In practice, it commonly uses a larger
number of negative samples than the number of supervised classes. However,
there is an inconsistency in the existing analysis; theoretically, a large
number of negative samples degrade classification performance on a downstream
supervised task, while empirically, they improve the performance. We provide a
novel framework to analyze this empirical result regarding negative samples
using the coupon collector's problem. Our bound can implicitly incorporate the
supervised loss of the downstream task in the self-supervised loss by
increasing the number of negative samples. We confirm that our proposed
analysis holds on real-world benchmark datasets.
