Generalized Category Discovery
  In this paper, we consider a highly general image recognition setting
wherein, given a labelled and unlabelled set of images, the task is to
categorize all images in the unlabelled set. Here, the unlabelled images may
come from labelled classes or from novel ones. Existing recognition methods are
not able to deal with this setting, because they make several restrictive
assumptions, such as the unlabelled instances only coming from known - or
unknown - classes and the number of unknown classes being known a-priori. We
address the more unconstrained setting, naming it 'Generalized Category
Discovery', and challenge all these assumptions. We first establish strong
baselines by taking state-of-the-art algorithms from novel category discovery
and adapting them for this task. Next, we propose the use of vision
transformers with contrastive representation learning for this open world
setting. We then introduce a simple yet effective semi-supervised $k$-means
method to cluster the unlabelled data into seen and unseen classes
automatically, substantially outperforming the baselines. Finally, we also
propose a new approach to estimate the number of classes in the unlabelled
data. We thoroughly evaluate our approach on public datasets for generic object
classification including CIFAR10, CIFAR100 and ImageNet-100, and for
fine-grained visual recognition including CUB, Stanford Cars and Herbarium19,
benchmarking on this new setting to foster future research.
