{'fraction_non_alphanumeric': 0.03944840972966415, 'fraction_numerical': 0.014012172189983218, 'mean_word_length': 4.601132502831257, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Revisiting Unsupervised Meta-Learning: Amplifying or Compensating for the Characteristics of Few-Shot Tasks

Meta-learning becomes a practical approach towards few-shot image classification, where a visual recognition system is constructed with limited annotated data. Inductive bias such as embedding is learned from a base class set with ample labeled examples and then generalizes to few-shot tasks with novel classes. Surprisingly, we find that the base class set labels are not necessary, and discriminative embeddings could be meta-learned in an unsupervised manner. Comprehensive analyses indicate two modifications -- the semi-normalized distance metric and the sufficient sampling -- improves unsupervised meta-learning (UML) significantly. Based on the modified baseline, we further amplify or compensate for the characteristic of tasks when training a UML model. First, mixed embeddings are incorporated to increase the difficulty of few-shot tasks. Next, we utilize a task-specific embedding transformation to deal with the specific properties among tasks, maintaining the generalization ability into the vanilla embeddings. Experiments on few-shot learning benchmarks verify that our approaches outperform previous UML methods by a 4-10% performance gap, and embeddings learned with our UML achieve comparable or even better performance than its supervised variants.

The primary antecedent of meta-learning is a labeled base class set (a.k.a. meta-train set), from which episodes of tasks are sampled to mimic the target few-shot scenario [55,12]. Labeled Instances Unlabeled Instances Figure 1: Illustrations of the difference between supervised and unsupervised meta-learning for few-shot image classification, respectively. There are no base class labels in the unsupervised case.

Supervised Meta Learning
In particular, each few-shot support set of a task is coupled with a query set with the same set of labels, so that the quality of the task-specific model could be evaluated. The metamodel could also be initialized through pre-training a large way classification task over such base class set [43,63,51].
To further reduce the labeling requirement in the holistic few-shot learning (FSL) process, we investigate whether we can meta-learn a few-shot classification model successfully without any labels during meta-training, which bypasses the labeling cost of the base classes. We can keep the class notion in the unsupervised scenario by following the exemplar perspective, where two instances come from the same pseudo class only when they are augmented from the same anchor. Thus, the embedding is meta-learned with pseudo classes in a contrastive manner, which facilitates the target few-shot tasks with the nearest neighbor search. Empirical observations indicate directly applying such an extension of supervised FSL methods usually gets poor results [2,22,44].
We take a closer look at unsupervised meta-learning (UML) via making a comprehensive empirical study and analyzing the key factors during meta-training. We observe that sufficient episodic sampling and right distance metric matters. To take full advantage of instances in a mini-batch, multiple few-shot tasks are re-sampled efficiently, and a semi-normalized similarity is used as the distance metric, which softens the logits adaptively. The simple modifications construct a strong baseline, which demonstrates 3-8% performance superiority over few-shot classification benchmarks. Furthermore, given only 10% base class labels to fine-tune the learned unsupervised baseline, the few-shot classification accuracy achieves nearly its supervised upper-bound.
Furthermore, we propose to improve the discriminative ability of the UML embeddings by taking account of task characteristics. First, we amplify the contrastiveness of instances by synthesizing hard tasks. Specifically, we augment the support set dynamically through mixing-up the ambiguous query instances with the nearest ones belonging to different pseudo classes. On the other hand, we append the embedding with a task-specific transformation. Such a special projection head takes in the specific property of tasks and leaves the vanilla embedding backbone more generalizable. Experiments on both few-shot classification and cross-domain benchmarks show our UML methods get promising performance compared with other unsupervised, self-supervised, or even supervised methods.
The remaining parts start with related work and preliminaries. After a close study of UML, we proposed our polished methods. Last are experiments and conclusion.

Related Work
Training a high-quality visual system usually requires an ample number of annotated training set with many shots per category [46,25], and directly training a model over few-shot classes is prone to over-fit. Few-shot learning (FSL) proposes to transfer the inductive bias from a labeled data-rich base class set to a few-shot task with novel classes [31,12,34,39,30,1]. Meta-learning becomes an effective tool for FSL, and one main thread of it meta-learns discriminative feature embeddings [49,41,47,56,29,63], so that with the help of non-parametric nearest neighbor classifier, novel class instances could be recognized given a few labeled examples. FSL has been achieved promising results in various domains [52,10,21,35]. Empirical studies of FSL could be found in [6,53].
Although FSL reduces the number of labels during the deployment of the system on novel classes, it still requires a large number of labels during the meta-model training. An intuitive question is whether we could still train a metamodel with limited or even no base class labels [2]. Benefited from semantic consistency among the perturbated views of a single instance, a meta-learning model could be trained by treating augmented versions of an instance as if they are in the same pseudo class. Both embedding-based [19] and optimization-based [44] methods are investigated. In this paper, we first construct a strong UML baseline with the help of sufficient sampling and semi-normalized distance metric. Then we propose two threads of methods to increase the discriminative ability of the UML embeddings.
Self-supervised learning is another possible way of learning an embedding model without labels [38]. Based on the pre-text tasks, the representation of objects becomes more discriminative and generalizable to "downstream" tasks [9,40,42,64,14,28]. Inspired by the similarity between the embedding-based meta-learning and the contrastive self-supervised learning methods [15,4], several recent approaches apply self-supervised learning in both supervised [50] and unsupervised meta-learning [19,33,37]. Although self-supervised learning methods perform similar to meta-embeddings, specific properties of tasks are not fully considered. In this paper, we utilize the characteristic of tasks in meta-learning to make further improvements.

Preliminary
We first review the supervised embedding-based metalearning for few-shot learning (FSL) and then describe how to extend them for unsupervised meta-learning (UML).

Supervised Meta-Learning for FSL
A task is a couple of support and query sets. We formalize the support set S = {(x i , y i )} N K i=1 in the K-shot N -way form, which contains N classes and K training examples per class. x i ∈ R D is an instance and y i ∈ {0, 1} N is its one-hot label. A model learned on S is evaluated on the query set Q with instances from the same distribution as S.
Instead of training a model over S with large K, FSL faces the challenge in transferring knowledge across learning visual concepts from a base labeled set B (a.k.a. metatrain set) to the non-overlapping novel classes (a.k.a. metatest set). Meta-learning mimics the target few-shot task by sampling tasks from B [55,12,49]: Episodes of K-shot N -way support sets S are sampled from B by randomly choosing N categories in B and K examples for each of the category. A meta-model f makes prediction on a query instance in Q with the same set of N classes as S, which is expected to be transferred to tasks with novel classes. The loss (·, ·) measures the discrepancy between the prediction and the true label. The meta-model training and inference stages on the base and novel class sets are called "meta-training" and "meta-val/test", respectively. Specifically, one effective implementation of the f in Eq. 1 is the prototypical network [49], which instantiate the inductive bias shared between base and novel classes as the d-dimensional instance embedding, i.e., f = φ : R D → R d . Denote y i = n selects instances in the n-th class, so the center of n-th support set class is p n = 1 K yi=n x i . The label of a query instance x j is predicted based on the nearest  center classifier with similarity measure Sim(·, ·) with φ: Sim(·, ·) in Eq. 2 is implemented as the negative distance. Another commonly used metric for embedding-based prediction is the cosine similarity [55] cos The joint optimization of Eq. 1 and Eq. 2 learns the embedding φ in a contrastive manner -visually similar instances are pulled to its center, while dissimilar ones are pushed away. Therefore it helps few-shot tasks with novel classes.

Unsupervised Meta-Learning for FSL
Sampling support and query sets in Eq. 1 require a base class set with a large amount of annotated examples. Benefited from the semantic consistency among the random augmentation of images, the supervised meta-learning could be extended to an unsupervised manner [2,44]. In particular, we treat an instance and its augmentations (e.g, random crop, horizontal flip) come from the same pseudo class. In this exemplar view, the sampled K-shot N -way support set (as well as query set) actually are the K random copies of N different images. We empirically find such a direct extension of supervised meta-learning methods works well in the unsupervised scenario once equipped with simple modifications.

Empirical Analysis on UML
We first investigate the episodic sampling and distance metric for unsupervised meta-learning (UML), which results in strong baselines. Then we empirically analyze UML and explore the influence of key factors in meta-training.

Simple Modifications towards Effective UML
Sufficient Episodic Sampling (SES). SGD is usually applied to optimize the meta-learning objective in Eq. 1, and there is one single task sampled in an episode [49,63]. In other words, a gradient descent step over the embedding φ is carried out after sampling each couple of N -way K-shot support and query sets. To better estimate the expectation, [12,13,32] accumulate the gradient and delay the backward operation after sampling multiple episodes, which requires high memory and is time-consuming We propose an efficient implementation of the metatraining update with almost no additional computation. Assume we sample a C-way K-shot task S ∪ Q in an episode where C > N , then we compute the embeddings of all instances with a single forward. Different from computing losses with the extracted embeddings on S and Q, we randomly re-split the instances into couples of support and query sets. Specifically, N of C classes are sampled, and K instances from each class are randomly selected into the support set. By repeating this process multiple times, we construct many pseudo tasks in one single episode, and one gradient descent is executed for the average of multitask losses. Experiments show Sufficient Episodic Sampling (SES) improves the efficiency of meta-training a lot, which fully utilizes the sampled instances in a mini-batch. Semi-Normalized Similarity metric (SNS). The similarity metric in Eq. 2 plays an essential role. As shown in [58], the cosine similarity works better than the negative Euclidean distance. [41,63] also point out the importance of logit temperature when using the Euclidean distance. We propose a semi-normalized similarity metric where only the support centers are normalized during meta-training: Since the similarity between the query instance x j and multiple support centers {p n } are compared simultaneously, missing the normalization on x j in SNS leads to the same predic-  tion with the cosine similarity. However, Eq. 4 amounts to utilize a dynamic temperature φ(x j ) for x j : The instance-specific temperature softens the logit value well and improves the efficiency of meta-training. The workflow for our UML baseline could be found in the supp.

Key Factors for Unsupervised Meta-Learning
We explore how to train an effective UML model through comprehensive empirical evaluations. Following the configuration in [22], we implement φ with a 4-layer ConvNet and focus on the MiniImageNet [55] with the standard split. The initial learning rate is set to 0.002, and is cosine annealed over 100 epochs. 64 instances are sampled in each mini-batch, and each instance is augmented into 3 copies to construct pseudo classes. 64-way 1-shot tasks are sampled for meta-training with the help of pseudo classes by default. We only use labels to evaluate some statistics such as few-shot classification accuracy, and no base class labels are utilized during meta-training. The same phenomenon is also observed on other datasets with deeper backbones. More details are in Section 6 and supp. Will sufficient sampling and semi-normalized metric help? Equipped with Semi-Normalized Similarity (SNS) Metric, we evaluate the Sufficient Episodic Sampling (SES) by gradually increasing the number of re-sampled tasks in one episode from one (denoted as the "vanilla" case) to many tasks. The change of meta-training loss and metavalidation 1-shot 5-way accuracy over sampled few-shot tasks are plotted in Fig. 2 (a)-(b). It is obvious by optimizing over more tasks in one episode, the model converges faster and generalizes better. There exists a huge gap over both the meta-training loss and meta-validation accuracy between the 512-task case and the vanilla one, which verifies the importance of sufficient sampling of tasks during meta-training. Furthermore, we find monotonously increasing the number of tasks has no additional improvements. So in our following experiments, we set the number of pseudo tasks to 512.
We also compare different similarity metrics, namely, cosine similarity, (negative) euclidean distance, inner product, and the proposed SNS. To fully explore the discriminative abilities of the metrics, we test the UML model over 5-way {1, 5, 20, 50}-shot novel class tasks. The few-shot classification accuracy on meta-test set is shown in Fig. 2 (c). Different from the observation in the supervised metatraining where the cosine similarity performs better than the Euclidean distance [58], in the UML case, the later metric show superiority than the former one in more shot scenarios. The proposed SNS outperforms other metrics when tested with different configurations of tasks. Effective configurations of tasks. For each episode, we sample C = 64 instances and make K + Q random augmentations for each of them. The batch-size is C × (K + Q), and 512 N -way K-shot tasks are sampled from the minibatch with SES. With fixed batch-size 256, there exists a trade-off among the factors such as N , K, and Q. We first observe that constructing tasks with larger ways works better. In other words, we sample N -way K-shot tasks from the mini-batch by increasing N as large as C. Similar to the supervised scenario [49], although it decreases the number of shots and query instances in a task, the larger-way tasks promotes the diversity of instances and facilitates UML. As shown in Fig. 3 (a), sampling 1-shot tasks with more ways consistently improves the few-shot classification accuracy.
Different from the higher-shot meta-training experience in supervised case [36,29], we find in UML the 1-shot metatraining performs the best on average. Fig. 3 (b) shows that the 5-shot meta-training (fix N = 64) improves the 20-shot and 50-shot meta-test tasks but degenerates the performance over 1-shot tasks. By fixing 64-way and 1-shot support set, we also investigate the number of query instances in Fig. 3 (c). Different from the influence of the shots, we find a larger number of query instances (and a larger batch-size) helps. Limited labeled base class instances leads to a strong FSL model. Based on the learned UML model, we can fine-tune its weights with a limited number of labels in the meta-training set following the supervised meta-learning 0% 1% 2% 5% 10% 20% 50% label ratio 46   manner. Results are shown in Fig. 4. We control the number of labeled instances in each base class. The zero label ratio corresponds to the fully unsupervised case, which is utilized as the initial weights for further analysis. By fine-tuning the weights with more labeled base class instances, the few-shot classification accuracy on the meta-test set increases and even exceeds the supervised ProtoNet [49] (the dotted line in the figure). The phenomenon indicates that a strong few-shot classification model only depends on a small amount of labeled base class once given a large unlabeled dataset. Similar observations are also proposed in [5] using self-supervised learning methods are the pre-trained weights for supervised classifier fine-tuning. We verify the importance of the UML model helps in the meta-learning field. Episodic training works better than contrastive selfsupervised learning baselines. Since the model is metalearned in an unsupervised way, an intuitive question is whether we can utilize the contrastive self-supervised learning methods to get similar results. In Fig. 5, we compare our UML baseline model with two representative contrastive learning methods, i.e, MoCo [15] and SimCLR [4]. Al-though these two comparison methods show promising results on self-supervised learning problems, there still exit performance gaps when compared with our episodic trained embedding. Hence the specific consideration of task characteristics is essential in the meta-learning problem.

Amplifying or Compensating for the Characteristics of Tasks for UML
The empirical analyses in Section 4 indicate considering the characteristics of tasks is essential in unsupervised metalearning (UML). Not only the sufficient sampling of tasks constructs strong baselines but also the explicit modeling of tasks in episodic training works better than the contrastive representation learning. To further improve the discriminative ability of the UML embeddings, we propose to take advantage of the characteristics of tasks from two aspects.

Adaptive Configuration of Difficult Tasks via Hard Mixed Supports
Through learning the similarity between each query and support instances with embedding φ, the meta-learning objective in Eq. 2 works in a contrastive manner so that query instances will be close to its corresponding support centers and far away from others. We try to amplify the characteristics of tasks by Hard Mixed Supports (HMS). To be specific, HMS dynamically constructs more complicated tasks with support distractors. Thus embedding φ is required to be more discriminative to discern similar instances.
Given a tuple of support and query sets (S, Q), we synthesize hard distractors for each query instance x j ∈ Q by mixing-up x j with its nearest neighbor x i ∈ S ∪ Q with different pseudo classes. Formally, we find K nearest neighbors of x j in the embedding space: Sim(·, ·) is SNS as Eq. 4. c is pseudo class. argmax K selects the top-K instances in the batch with the highest similarity. To increase the difficulty, we further mix-up the query instances with the hard mined neighborsŜ j of x j , i.e, λ ∈ [0, 0.5] is a random value sample from a uniform distribution [20]. Eq. 7 interpolates distractors between the query instance embedding φ(x j ) with the embedding of its hard mined neighbor φ(x). The strength of the mix-up coefficient λ is controlled so that the mixed instances are biased towards the mined neighbor, which guarantees the semantic space is not messed up. Finally, we augment the original one S with the mixed support setS j . Denote the query instance-specific support embedding set To relieve the variety of task distribution between base and novel classes, task specific projection head (implemented with Transformer [54]) is used to transform task agnostic embeddings to task specific ones. This component is discarded once we finish meta-training.
re-compute the loss function in Eq. 1 over S j and optimize the model with back-propagation. HMS is operated over the embeddings, so it incurs a negligible computational burden.
In summary, we augment the support set with distractors for each query instance, which is equivalent to dynamically constructing a hard support set for each query. The more difficult support set amplifies the characteristic of tasks, and facilitate to learn more discriminative embeddings.

Task-specific Projection Head
UML suffers from differences of task distribution between base and novel classes. The unsupervised tasks sampled from the unlabeled data in meta-training could be very different from those supervised tasks constructed according to labels in meta-test. Since we construct unsupervised meta-training tasks with data augmentation, it is possible that two instances from different pseudo classes actually come from the same supervised category. Requiring the embedding to distinguish between them will mislead models to push instances from the same supervised category away, which degrades the discriminative ability of embeddings. It is inevitable that we assign semantically similar instances into different pseudo classes in UML tasks. To compensate for the characteristics of tasks, we propose to decouple the model into generalizable and specific components by learning an additional Task-Specific Projection Head (TSP-Head) with a set-to-set function [63]. Therefore, the specific properties in a task will be captured by the top-layer transformation. The embeddings could be more generalizable and facilitate novel class few-shot tasks during the meta-test phase.
During meta-training, support and query embeddings in a task are transformed with a set-to-set function T: We measure the similarity between query and support in Eq. 4 with the transformed embedding ψ(x) instead of φ(x) in meta-training, while during meta-test only φ(x) is used. We implement T(·) with Transformer [54]. Given an instance embedding φ(x) in the set {φ(x)} x∈S∪Q , its output is adapted based on a "key-value" dictionary module. In particular, denote W Q , W K , and W V as three d × d projections, and the affinity between x with another instance x i in the set is measured as the projected inner product The transformed embedding ψ(x) is a weighted sum over another transformed set of the embeddings L(·) is a sequential operation of layer normalization, dropout, and linear projection. The adapted embedding takes a holistic consideration of all other embeddings in the task, so that the specific properties of the task is stressed, which leaves the embedding φ more generalizable. The negative influence of the semantically similar instances from different pseudo classes are relieved as well. Moreover, the transformer could be processed multiple times (a.k.a. multi-layer). More than one set of projection matrices could be allocated, and the multiple adapted embeddings could be concatenated followed by a linear projection to dimensionality d (a.k.a. multi-head). In our empirical study, we find multi-head version works better, but using more layers will not help. Details could be found in Section 6 and the supplementary. Different from FEAT [63] where the set-to-set transformation is utilized as a component in the meta-model to obtain task-specific embeddings for few-shot classification, here we use the set-wise transformation function as a projection head to compensate for the characteristic of different tasks, and only the embedding φ is used during meta-test.

Experiments
We evaluate our UML baseline, HMS, and TSP-Head on few-shot classification benchmarks. The generalization ability on novel domains and the increase of performance given limited base class labels are also investigated.

Experimental Setups
Datasets. MiniImageNet [55] contains 100 classes and 600 images per class. 64/16/20 non-overlapping splits are used as meta-training/validation/test [45], respectively. All images are cropped to 84 × 84. Two splits of CIFAR-100 [29] are investigated with 32 by 32 small images. In the CIFAR-FS, 64/16/20 classes are selected from the 100 classes for meta-train/val/test. While for FC-100, 60, 20, and 20 classes are selected in a special manner to enlarge the discrepancy among the three sets. Each class contains 600 images. No meta-training labels are used by default. Evaluation protocols. We follow the classical protocol to evaluate the meta-learned model on few-shot classification tasks [55,49,63,58]. 10,000 N -way K-shot tasks are sampled from the meta-test set during the model evaluation, and there are 15 instances for each of the N class in the query set. The mean accuracy and 95% confidence interval are reported. We set N = 5 and K = {1, 5, 20, 50}. Implementation details. We implement φ with two representative backbones. The ConvNet backbone [55,49,12] has 4 sequential concatenation blocks with convolution, batch normalization [18], ReLU, and Max Pooling. Each of the blocks outputs 64-dimensional latent embeddings, and we append a global average pooling at last. We also consider a 12-layer residual network [16,29]. We apply ADAM [23] on ConvNet with initial learning rate 0.002 over 100 epochs, and SGD w/ momentum on ResNet with initial learning rate 0.03 over 200 epochs. For each epoch, we perform 100 episodes. Cosine annealing is utilized to tune the learning rate. To generate different augmented versions of an instance during meta-training, we follow [22] and take advantage of a composition of the random resized crop, random translate, color distortions, and random grayscale. Top-10 nearest support instances are selected in HMS to construct difficult distractors. A 1-layer 8-head transformer is used in TSP-Head. Code will be made available.

Comparisons on Benchmarks
The average 5-way {1,5,20,50}-shot classification accuracy on MiniImageNet, CIFAR-FS, and FC-100 with Con-vNet and ResNet backbones are listed in Table 1 and Table 2, respectively. The confidence interval values are in the supp. The UML baseline utilizes sufficient episodic sampling (SES) and semi-normalized similarity (SNS) metric. Based on this, we apply hard mixed supports (HMS) and task-specific projection head (TSP-Head).
Three blocks in Table 1 list results of the compared UML methods, self-supervised learning methods, and our approaches. Our baseline already outperforms previous methods with a large margin -for example, an 8-10% accuracy superiority when compared with UMTRA [22]. HMS and TSP-Head achieve further improvements, where TSP-Head performs better especially when evaluated with more shots.

Cross-Domain Comparisons
Since the embeddings are learned in an unsupervised manner, it could be memoryless on the base-class information and become more generalizable on novel domains. With the embedding meta-learned on the base class set of MiniImageNet, we evaluate it on CUB [57]. CUB is a finegrained dataset on different species of birds. Following the splits of [63], 50 classes are randomly selected from CUB as the meta-test set, where 5-way {1,5,20}-shot tasks are sampled for embedding evaluation in Table 3. Results verify improvements on the in-domain evaluation also generalize to novel domains. TSP-Head gets the best results in all cases.

Ablation Studies
We further investigate the properties of the UML learned embedding on MiniImageNet with ResNet backbone. More analyses with different evaluation criteria are in the supp. Comparison with the supervised upper-bound. The embedding learned without base class labels show strong discriminative ability on benchmarks. We compare UML embedding with its supervised upper-bound in Table 4.
Similar to Fig. 4, we also evaluate the influence of the label ratio on the ResNet backbone. The initial weights of the embedding are trained on the whole base class set without labels. When we gradually increase the number of base baseline HMS TSP-Head  class labels, we fine-tune the weights with supervised metalearning techniques over the limited labeled base class set. The results show that UML pre-trained model can be tremendously label-efficient as is revealed in [5]. Surprisingly, we also find that based on UML pre-train and with only 1% labels, classical meta-learning method can achieve performance comparable to fully supervised model. Pre-trained with HMS, the fine-tuned model with 10% labels can even be competitive to a number of state-of-the-art supervised methods using supervised pre-train with full labels. Is task-specific projection head necessary? The projection head is a useful component in self-supervised learning, e.g, SimCLR [4], where a nonlinear projection layer is ap- pended over the top-layer embedding. We also implement the projection head as a task-agnostic one, and the results are in Table 5. The advantage of the TSP-Head indicates the explicit consideration of the specific properties of sampled tasks during meta-training is necessary. Combine two techniques together? We propose two methods to take the characteristics of tasks into account. An intuitive question is whether the two methods could be fused to get further improvements. As shown in Table 6, we find HMS and TSP-Head are not compatible, and directly combining them cannot get better results.

Conclusion
Instead of learning to learn embeddings for few-shot classification through a labeled base class set, we propose to transform the meta-learning methods to a fully unsupervised manner from both sampling and modeling aspects. Simple modifications like sufficient sampling and semi-normalized distance lead to strong Unsupervised Meta-Learning (UML) baselines. We further consider a strategy constructing hard meta-training tasks dynamically and utilizing task-specific projection heads to decompose the model's generalizable components into embeddings. Both approaches take full advantage of the characteristic of tasks. Our UML methods outperform other UML models on few-shot classification benchmarks, and they also achieve similar state-of-the-art performance with its supervised variants given only 10% of base class labels. Further studies include exploring other architectures to decompose the characteristic of tasks.

A. Towards Unsupervised Meta-Learning
After a brief review of the Unsupervised Meta-Learning (UML) for Few-Shot Learning (FSL), we present our proposed UML baselines, Hard Mixed Supports (HMS), and Task-Specific Projection Head (TSP-Head) in detail with more analyses.

A.1. Unsupervised Meta-Learning for FSL
We define a task as a couple of support set S and query set Q, which are in the K-shot N -way form and the Qshot N -way form, respectively. For example, there are N classes in the support set and each class has K instances, i.e, The instances in the query set Q come from the same set of N classes in S. The target of few-shot learning is to learn a model from S with small K, which has high discerning ability on Q.
Directly training a model such as a deep neural network on the support set is prone to over-fit. In UML, we collect unlabeled instances from related domains non-overlapping w.r.t. the N classes, which constructs a base class set B (also known as meta-training set). In other words, we first learn instance embedding from the base class set, and the embedding acts as an inductive bias that generalizes to the target few-shot tasks with novel classes. Therefore, we can achieve low classification error when we apply similarity-based classification with the learned embedding. We denote the two phases, learning and evaluating the learned embeddings, as meta-training and meta-test, respectively.
To mimic the target few-shot classification, we sample episodes of tasks from the unlabeled base classes. Since there are no labeled as the supervised case, we construct pseudo classes via image augmentation (e.g, random crop, horizontal flip). During meta-training, we sample episodes of pseudo labeled tasks (S, Q) ∼ B in the following way: we randomly select N instances from the base class set, and then make K copies of them to get a K-shot N -way support set, another Q copies of each instance constructs the query set. The embedding f = φ : R D → R d is learned in the following objective: The loss (·, ·) measures the discrepancy between the prediction and the true label. A meta-model f makes prediction on a query instance in Q with the help of φ: for all (x j , y j ) ∈ Q do 6: Get f (x j , S) with selected metric 7: Compute (f (x j , S), y j ) 8: end for 9: Accumulate loss for all x j as Eq. 11 10: Update φ with SGD 11: end for 12: return Embedding φ Algorithm 2 The meta-test flow of UML.
Require: A labeled support set S and an unlabeled query instance x j from non-overlapping class w.r.t. B, the learned embedding φ 1: Compute p n for all n = 1, . . . , N classes 2: Compute Sim(φ(x j ), p n ) with selected metric and φ 3: Predict via arg max n Sim(φ(x j ), p n ) 4: return The predicted label of x j We follow the prediction manner in prototypical network [49], where Sim(·, ·) measures the embedding of a query instance x j with each class center p n = 1 K yi=n x i in the support set. We use y i = n selects instances in the nth class. In this way, the larger the similarity between a query instance with a support center, the larger the probability the query instance comes from the corresponding class.
The similarity could be measured in various metrics, for example, the negative distance: the cosine similarity: or the inner product Although no labels of the base class are utilized, the joint optimization of Eq. 11 and Eq. 12 learns the embedding φ in a contrastive manner -visually similar instances are pulled to its center, while dissimilar ones are pushed away. Therefore it helps few-shot tasks with novel classes. The meta-training and meta-test phases of the vanilla UML is described in Alg. 1 and Alg. 2, respectively. Apply K + Q augmentations for all C instances 4: Get φ(x) for all augmentations 5: for all task iter = 1,... do 6: Split instances to get a N -way K-shot task (S, Q) 7: for all (x j , y j ) ∈ Q do 8: Get f (x j , S) with SNS in Eq. 16 9: Compute (f (x j , S), y j ) Accumulate loss as Eq. 11 for all tasks 13: Update φ with SGD 14: end for 15: return Embedding φ

A.2. The proposed UML Baseline
We propose two modifications for UML meta-training from both the sampling and modeling aspects, i.e, the Sufficient Episodic Sampling (SES) and the Semi-Normalized Similarity (SNS) metric. The vanilla method becomes a strong UML baseline.
SES samples multiple few-shot tasks in one episode efficiently, which estimates the embedding gradient more accurately. As listed in Alg. 3 line 5-6, based on the extracted features for the augmented instances, we randomly split all instances into different support-query sets as multiple tasks. The average loss over all tasks is computed for the embedding update. Since only one forward operation of φ is applied (line 4), SES has negligible additional computational costs during meta-training. In our implementation, we set C = N , so our baseline has the same mini-batch size as the vanilla method.
SNS measures the similarity from the query instance embedding to the support center in the following way: Due to the normalization of softmax in Eq. 12, our SNS makes the same prediction with the cosine similarity in Eq. 14. However, it adapts the scale of the logit for each instance with the norm φ(x j ) , and improve the discriminative ability of the embedding during the meta-training. The usage of SNS is shown in Alg. 3 line 8. We empirically analyze the difference between the metrics. Based on the standard MiniImageNet, we train unsupervised meta-learning models as Eq. 11 with a 4-layer ConvNet. 512 tasks are used in SES, and we only change the metrics among cosine similarity, inner product, (negative) Euclidean Sample C instances from B

3:
Apply K + Q augmentations for all C instances 4: Get φ(x) for all augmentations 5: for all task iter = 1,... do 6: Split instances to get a N -way K-shot task (S, Q) 7: for all (x j , y j ) ∈ Q do 8: Sample λ ∼ U (0, 0.5) Get f (x j , S j ) with SNS in Eq. 16 13: Update φ with SGD 18: end for 19: return Embedding φ distance, and our SNS. The norm of the embedding gradient, i.e, ∇ φ(x) 2 averaged over all instances in the mini-batch, is recorded along with the training progress, and the results are shown in Fig. 8. We find the cosine similarity has large gradient norm during the meta-training. Since the cosine value varies near 0 in the range [-1, 1], which results in some steps on norm changes during meta-training. The inner product and the Euclidean distance lead to smaller gradient norm. Our SNS, however, has the smallest norm value when compared with others. We conjecture by using different similarity measures, the embedding φ is updated in diverse manners. SNS makes φ updated directly towards the stable point, while others such as the inner product make aggressive updates at first, and then gradually adapts the embedding to the target solution. Based on the meta-test classification performance, SNS converges faster and generalizes better.

A.3. Further Improvements for UML
Based on our UML baseline, we amplify or compensate for the characteristics of tasks during meta-training, with the Hard Mixed Supports (HMS) and Task-Specific Projection Head (TSP-Head).
HMS first selects those instances difficult to decided with ambiguous confidences, and then mix their embeddings with the embeddings of instances in the support set. Finally, the hard mixed support instances are added to the original support set, which increases the difficulty of the task and amplifies the characteristic of the task. The main flow of for all (x j , y j ) ∈ Q do 9: Get f (x j , S) with SNS and ψ(x) in Eq. 16 10: Compute (f (x j , S), y j ) Accumulate loss as Eq. 11 for all tasks 14: Update φ and T with SGD 15: end for 16: return Embedding φ HMS is listed in Alg. 4 (the changes w.r.t. our baseline model are marked with blue).
From another perspective to take advantage of the characteristic of tasks, we consider compensating for it via including a task-specific projection head during meta-training. The projection head is a kind of embedding transformation, which adapts the embedding φ to ψ based on the context (both the support and query set instances) of the task. Therefore, the specific properties in a task will be captured by the top-layer transformation. The embeddings could be more generalizable and facilitate novel class few-shot tasks during the meta-test phase. Furthermore, the set-wise transformation also lowers the risk of selecting semantic-similar pseudo class instances in a mini-batch. The main flow of TSP-Head is listed in Alg. 5 (the changes w.r.t. our baseline model are marked with blue).
Note that both HMS and TSP-Head change the metatraining flow, and only the learned embedding φ is used during meta-test.

B. Empirical Analysis on UML
Same as the analyses in the main paper, we investigate the key factors in unsupervised meta-learning (especially the meta-training). Following the configuration in [22], we implement φ with a 4-layer ConvNet and focus on the MiniImageNet [55] with the standard split. The initial learning rate is set to 0.002, and is cosine annealed over 100 epochs. We only use labels to evaluate some statistics such as few-shot classification accuracy, and no base class labels are utilized during meta-training. The detailed configurations and conclusions are in the captions of Fig. 9 to Fig. 11.

C. Detailed Experimental Configurations
Network architectures. We implement φ with two representative backbones. The ConvNet backbone [55,49,12] has 4 sequential concatenation blocks with convolution, batch normalization [18], ReLU, and Max Pooling. Each of the blocks outputs 64-dimensional latent embeddings, and we append a global average pooling at last. We also consider a 12-layer residual network [16,29], which is denoted as ResNet in the experiments.
Optimization. We apply ADAM [23] on ConvNet with initial learning rate 0.002 over 100 epochs. For ResNet, we follow [29,63] and use SGD w/ momentum 0.9 over 200 epochs, whose initial learning rate is 0.03. For each epoch, we perform 100 episodes. Cosine annealing is utilized to tune the learning rate for both architectures. We observe by increasing the number of ways we can get more discriminative embeddings on more shot evaluations, and further increase of the way value N does not show apparent contribution when N exceed some threshold. One explanation for this phenomenon is that the number of ways is related to the difficulty of the sampled tasks during meta-training. The more difficult the training process is, the better model we have.
Augmentations. We take advantage of a composition of the random resized crop, random translation, color distortions, and random grayscale to construct the pseudo labels for unsupervised meta-training.
• Random resized crop makes a crop of random size (uniform from 0.08 to 1.0 in area) of the original size and a random aspect ratio (default: of 3/4 to 4/3) of the original aspect ratio. Then the uncovered blank area is filled with reflect padding.
• Random translation translates vertically and horizontally by n pixels where n is integer drawn uniformly and independently for each axis from [−4, 4]. Then the uncovered blank area is filled with reflect padding.
• Color distortions randomly change the brightness, contrast, and saturation of an image.
• Random grayscale randomly converts the image to grayscale with a probability of 0.25.
Similarity metrics. There are four different kinds of similarity metrics -cosine, inner product, (negative) Euclidean distance, and our proposed SNS in our investigations. Inner product and SNS range (−∞, +∞). Euclidean ranges that the number of query instances needs to reach a certain value in order to make accurate evaluation for few-shot classification. But too many query instances will not get further improvements.
(−∞, 0]. Unlike the former three, the range of cosine is [−1, 1], which makes it impossible to cover all the categorical distributions. To relieve this problem, a temperature τ is usually used to scale the logit [4,15]: (17) Following the practice in [15], we set τ = 0.5 for cosine similarity. For other similarity metrics, we do not use the scale value.
Fine-tuning. The learned UML embedding could be used as a pre-trained weights for supervised meta-learning, which reduces the number of labeled base class data a lot. We use the learned UML embedding φ as the initialization, and fine-tune the emebdding with Eq. 11. During the fine-tuning process, our proposed SES and SNS are applied, and we sample episodes of tasks based on the true label of instances.
In detail, we optimize 10 episodes for each epoch. In each episode, we sample a batch consisting of 256 images from 64 classes, 4 images for each class. Based on the sampled batch, 512 tasks are further sampled as in SES. The learning rate is set to 0.0001 for both architectures. We set the fine-tuning optimizers the same as the ones used in the UML phase, i.e, ADAM [23] on ConvNet and SGD w/ momentum on ResNet. We fine-tune the embedding for 50 and 100 epochs when label ratio below and above 10%, respectively.
Hard Mixed Supports (HMS). If not specified, HMS selects 10 nearest neighbors for each query. The embedding mix-up coefficient λ ∼ U (0, 0.5). We investigate the influence of the mix-up strength s based on which we sample λ ∼ U (0, s) in Section 6 and Table 7.
Task-Specific Projection Head (TSP-Head). TSP-Head uses a 1-layer and 8-head transformer by default [54]. The dimensionality of key, query and value vectors is the same as the input, i.e, 64 for ConvNet and 640 for ResNet.

D. Additional Experiments
We provide additional experimental results and analysis in this section.

D.1. Linear Evaluation
We also evaluate our UML methods following the classical self-supervised learning protocol. In detail, we freeze the feature extractor (i.e, the embedding) and train a linear classifier on top of it. After extracting the features of all instances in the meta-test set, we randomly split the meta-test set into 10 folds. Each fold has the same number of samples for each class. 9 of the 10 folds are used to train a linear logistic regression model, and we test the learned linear model on the remaining one fold. The logistic regression is trained with 2 regularization and the regularization weight is searched in a logarithmic scale between 10 −4 and 10 4 with 5-fold cross validation. We report test accuracy in Fig. 12. Two representative self-supervised method, MoCo-v2 [7] and SimCLR [4], are compared, whose hyper-parameters are consistent with their published papers. With this special evaluation protocol, we find although our UML baseline outperforms self-supervised learning methods on few-shot learning tasks, it does not necessarily have an advantage on other downstream tasks like linear classification. By equipping the baseline with HMS or TSPhead, it achieves better results as shown in Fig. 12, which indicates that through incorporating characteristics of tasks, the learned embeddings can be more generalizable to handle various kinds of tasks. The TSP-Head achieves the best results, which is consistent with our observation that TSP-Head facilitates more shot tasks. We hope the evaluations with few-shot tasks and with linear models will draw more insights on both the unsupervised meta-learning and the self-supervised learning.

D.2. Ablation Study on Architectures
The strength to mix-up embeddings in HMS. HMS creates hard tasks for each query not only by mining the most ambiguous supports, but also by mixing up the mined instances with the query. By default, the mix-up coefficient is sampled from a uniform distribution ranging from 0 to 0.5. We set the threshold as 0.5 to ensure the mixed embedding is biased towards the mined instance and semantically different from the query. We investigate the influence of different mix-up coefficient ranges, i.e, we sample λ ∼ U (0, s) with different s. Table 7 shows that with the mix-up strength becomes larger, HMS gets better few-shot classification accuracy, especially 1-shot and 5-shot. Therefore, larger mix-up strength  helps UML. We keep s = 0.5 in all our experiments, but with carefully selected s, HMS is able to get better results especially in 20-shot and 50-shot tasks as Table 7.
Transformer configurations in TSP-head. As we stated in the main paper, the transformer could be processed multiple times (a.k.a. multi-layer). More than one set of projection matrices could be allocated, and the multiple adapted embeddings could be concatenated followed by a linear projection to dimensionality d (a.k.a. multi-head). We investigate whether such more complicated versions of Transformer will help TSP-Head. The comparison results of different setups are shown in Fig. 13 and Fig. 14, respectively. Based on the results in Fig. 13, we find increasing the number of Transformer head does help learn better embeddings. While the results in Fig. 14   fore, we use 8-head 1-layer version of Transformer in TSP-Head. It is notable that the configuration of the Transformer is different from a recent supervised usage of Transformer for few-shot classification [63]. The experiments in [63] show only 1 head and 1 layer works the best with supervised meta-training, while we find more heads facilitate the unsupervised meta-learning case.

D.3. Visualize learned embeddings
We visualize the learned embedding φ with t-SNE on the meta-training and meta-test splits of MiniImageNet. For clarity, we randomly select 5 classes from meta-training and meta-test sets, respectively. In each class, embeddings of randomly selected 40 instances are plotted. Embeddings from meta-training set are marked as "Base", for the model has used them in meta-training. Those embeddings from metatest set are marked as "Novel". Different colors represent different classes. " " marks the center of the corresponding class.
The embeddings learned in a supervised manner present a good clustering effect on base classes. Samples from these classes can be separated easily. But the novel classes do not show good properties as the base ones, which are close to each other and can not be easily separated. On the contrary, our unsupervised methods show similar properties on both base and novel classes. They form kinds of clusters but do not overfit on base class data, which indicates that the embeddings learned in an unsupervised manner may have more generalization potential. Fig. 15(d) shows an obviously different representation learned by TSP-Head. Samples are scattered among the space, keeping a certain distance from each other. This kind of uniformity may explain why TSP-Head is good at 20 shot and 50 shot tasks.

D.4. Benchmark Results
We include the confidence interval over 10,000 trials when we evaluate UML methods on the meta-test set in Table 8.