{'fraction_non_alphanumeric': 0.03325147156278018, 'fraction_numerical': 0.010408139398900713, 'mean_word_length': 4.881247134342045, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Simultaneous Perturbation Method for Multi-Task Weight Optimization in One-Shot Meta-Learning

Meta-learning methods aim to build learning algorithms capable of quickly adapting to new tasks in low-data regime. One of the most difficult benchmarks of such algorithms is a one-shot learning problem. In this setting many algorithms face uncertainties associated with limited amount of training samples, which may result in overfitting. This problem can be resolved by providing additional information to the model. One of the most efficient ways to do this is multi-task learning. In this paper we investigate the modification of a standard meta-learning pipeline. The proposed method simultaneously utilizes information from several meta-training tasks in a common loss function. The impact of these tasks in the loss function is controlled by a per task weight. Proper optimization of the weights can have big influence on training and the final quality of the model. We propose and investigate the use of methods from the family of Simultaneous Perturbation Stochastic Approximation (SPSA) for optimization of meta-train tasks weights. We also demonstrate superiority of stochastic approximation in comparison to gradient-based method. The proposed Multi-Task Modification can be applied to almost all meta-learning methods. We study applications of this modification on Model-Agnostic Meta-Learning and Prototypical Network algorithms on CIFAR-FS, FC100, miniImageNet and tieredImageNet one-shot learning benchmarks. During these experiments Multi-Task Modification has demonstrated improvement over original methods. SPSA-Tracking algorithm first adapted in this paper for multi-task weight optimization shows the largest accuracy boost that is competitive to the state-of-the-art meta-learning methods. Our code is available online.


Introduction
One of the most important hallmarks of human intelligence is the ability to learn new concepts from few examples. However, despite significant progress in deep learning in a variety of fields in recent years limitations of deep learning approaches still exist in many practical applications where labeled data is intrinsically rare or expensive. For machine learning systems it is difficult to adapt to new concepts fast with very little supervision. That raises attention to the challenging few-shot learning problem and its most difficult setting: one-shot learning. One of the promising areas of research in recent years was to tackle this problem using the approach of meta-learning or "learning to learn" [1,2,3].
Meta-learning process is performed on a family of tasks set on disjoint meta-training and meta-testing sets. Each task includes only a limited amount of training data which requires fast adaptability of the meta-learner. The performance of the meta-learner is evaluated on meta-testing tasks. In this paper we have modified Multi-Task Modification (MTM) approach for meta-learning from [4] where this algorithm was investigated on Omniglot dataset, which is a less challenging setting than those discussed in this paper. Multi-task weight optimization plays the key role in MTM. Therefore, we consider members of the family of Simultaneous Perturbation Stochastic Approximation (SPSA) methods as optimizers of multi-task weights and compare them with a gradient-based approach. To the best of our knowledge, we have adapted SPSA for Tracking method for multi-task weight optimization in the one-shot meta-learning setting for the first time. To summarize, our main contributions are as follows: • We have successfully applied our MTM to the important optimization-based method MAML [1] and the important metric-based method Prototypical Networks [5]. For both of these methods, the proposed approach achieved an improvement over the original method on the main one-shot learning benchmarks.
• We have formulated novel SPSA for Tracking method (SPSA-Track) as a multi-task weights optimizer in MTM for meta-learning. This method demonstrated a significant performance boosting on average.
• We have shown robustness of SPSA-based approaches on multiple benchmarks. Also, they appear to outperform gradient-based methods as a multi-task weights optimizer.
• We have demonstrated that combining MTM SPSA-Track with modern backbone model achieves a significant boost in accuracy. Such a performance improvement puts our result among the best in the field.

Related Works
Few-Shot Learning. Meta-learning attempts to acquire general knowledge of a target domain by learning many tasks that lie within it [1]. Few-shot learning is widely used as one of the main benchmarks for meta-learning approaches [6,1,7]. In one-shot learning setting, training set consists of only one sample per task. It is expected that by training on similar yet different tasks the model will learn common features that will still be relevant to unseen tasks and, thus, acquire general understanding of the field. Few-shot learning models are typically divided into 2 broad categories based on how the problem is modelled: optimization-based and metric-based.
The class of optimization-based few-shot learning algorithms uses explicit optimization for fast adaptation to new tasks. Model-Agnostic Meta-Learning (MAML) [1] attempted to find network weights that are able to quickly adapt to new tasks through an optimization procedure. Many refinements to the MAML algorithm have been proposed since its inception, for instance, Latent Embedding Optimization (LEO) [3] tried to improve generalization by learning in a lower-dimensional latent embedding space. R2D2 [8] and MetaOptNet [2] improved accuracy by using Ridge Regression and SVM as classifiers.
Metric-based approaches are a class of methods for few-shot learning problems that aim to learn a discriminative embedding transferable to a target task. Metric learning has a long history of research and various applications [9,10]. Siamese Convolutional Neural Networks [11] was the first metric-based method for one-shot image classification that learned task-agnostic feature embeddings. Matching Networks [6] enhanced this approach by using two different networks and episodic training. Prototypical Networks (ProtoNet) [5] introduced the idea of learning class representation by using mean features embeddings. Two other important approaches were suggested in Relational Networks [12] that introduced an architecture to model the similarity of embeddings and Task Dependent Adaptive Metric (TADAM) [13] that switched to learning task-specific features. Recently, Deep Subspace Networks (DSN) [14] have been used to predict class labels by calculating the distance between a query point and its projections onto discriminative subspaces formed by the support sets for each class.
Transductive inference as an approach to the few-shot learning problem was the subject of research in several recent papers [15]. In this setting a classifier model performs class predictions jointly for all the unlabeled query examples of a single few-shot learning task, instead of making predictions for one sample at a time as in inductive methods. As an alternative to the meta-learning framework, several recent studies investigated performance of standard end-to-end pre-trained classifiers on few-shot tasks [16,7,15]. However, in this work we focus on improving the meta-training phase and consider only the inductive methods that use meta-learning pipeline without pre-training. For this reason, we selected MAML and ProtoNet because papers describing these methods are among the most cited in the field. Thus, we expect that results of applying multi-task meta-learning modification should generalize when used for the modification of other one-shot learning algorithms following these restrictions.
Multi-Task Learning. Multi-task learning is an approach where a model is trained to give predictions for multiple tasks jointly. Multi-tasking can be considered as an implicit data augmentation and regularization technique assisting the network to focus on more relevant features in the input [17]. However, proper task weighting might be crucial to improving model performance. Kendall et al. [18] have introduced a multi-task loss function that relies on maximizing the Gaussian likelihood with task-dependent uncertainty. The proposed single model has outperformed separate models for each task. In [4], multi-tasking approach has been applied for a few-shot character recognition problem, which resulted in an improvement over the baseline model. A close connection in terms of optimization task formulation between multi-task learning and gradient-based meta-learning was established in [19].
Stochastic Approximation. The stochastic approximation algorithm was developed for solving the optimization problem by Kiefer and Wolfowitz [20]. Simultaneous Perturbation Stochastic Approximation (SPSA) algorithm [21] uses only two observations at each iteration which recursively generates estimates along random directions. In highdimensional setting SPSA has the same order of convergence rate as Kiefer-Wolfowitz approach while requiring significantly fewer measurements of a function. When an unknown but bounded disturbance corrupts the observed data, the quality of methods based on stochastic gradient decreases. However, SPSA-like algorithms demonstrate a high level of resistance to such disturbances [22]. Stochastic approximation algorithms are successfully used in various machine learning problems [22].

One-Shot Learning Problem Definition
We consider one-shot learning problem as a special case of few-shot learning. According to the few-shot learning problem formulation, we need to train a classifier that can quickly adapt to new unseen classes using only few labeled examples of classes. To cast this problem as meta-learning problem, Vinyals et al. [6] proposed the pipeline where elements of each class were randomly divided into support set and query set. A set of classes in training phase does not overlap with a set of classes in testing phase. In the meta-learning pipeline, training and testing processes consist of a series of episodes where each episode ξ t includes one or more tasks and each task t i consists of support and query sets for several classes.
containing elements of all N C classes included in the task t i of per class support and query sets, respectively. Parameter M ∈ N can vary in different algorithms. For each task t i in an episode, L θ,ti (D) denotes the value of loss function on D, where θ is the vector of learnable parameters of model φ θ (convolutional neural network in this paper) and D can be support, query, or another dataset.
Model-Agnostic Meta-Learning. MAML uses loss function which is a two-step training procedure: 1) adaptation step (inner-loop) where network parameters are adapted to the specific task t i : 2) meta-gradient update (outer-loop) step where parameters θ are updated by backpropagating through the adaptation procedure: where α, β are adaptation and meta-learning rates correspondingly.
Prototypical Networks. In ProtoNet algorithm model φ θ computes embedding of input data. Each class k is represented by prototype vector c k ti which is calculated as a mean vector of the corresponding support set. The Prototypical Networks model φ θ is trained via stochastic gradient descent (SGD) by minimizing loss function for train task t i , where d(·, ·) is Euclidean distance function:

Multi-Task Meta-Learning Loss Function
Many few-shot learning methods [6,5,2] work with M = 1 (i.e. with one task in a training episode ξ t ). Other approaches [1,23] utilize a batch of tasks with M > 1 per episode but consider the contribution of each task equally. In this paper we develop the idea of adaptive use of information from multiple tasks simultaneously via the multi-task approach for meta-learning proposed in [4]. Multi-task learning methods for deep neural networks can be divided into two main areas: soft and hard parameter sharing of hidden layers of neural network [17]. In accordance with [4], we use hard parameter sharing for all hidden layers of a convolutional network. Thus, one neural network is used for all tasks, and the presence of several tasks is reflected only in the loss function. For this purpose [4] proposed an adapted approach discussed in [18] which used task-depended (homoscedastic) uncertainty as a basis for weighting losses in a multi-task learning problem.
Corresponding multi-task meta-learning loss function takes the following form: where weights ω t = (ω ) are hyperparameters.

Multi-Task Weights Optimization
As shown in [18], model performance is extremely sensitive to multi-task weights selection and tuning of ω t is critical for the success of a multi-task learning. On the other hand, searching for these optimal weights is expensive and increasingly difficult for a large model with numerous tasks.
In this work we focus on developing and investigating performance of optimization methods for hyperparameters ω t in the loss function (4). Nowadays gradient optimization approach is a natural choice in many deep learning algorithms [24,18]. Therefore, we consider an approach with embedding weights optimization in the backpropagation procedure similar to the one described in [18]. This procedure is described in Algorithm 1. Let ω t and θ t denote estimates at iteration t of multi-task weights and model parameters, respectively. This multi-task meta-learning modification pipeline for a training episode ξ t is visualized in Figure 1.
In this paper gradient-based and stochastic approximation methods are considered and compared as the multi-task weights optimizer.
When solving a meta-learning problem, algorithms face model uncertainties associated with a critical issue of limited amount of training samples per task, which may result in overfitting [25]. This phenomenon is especially pronounced in the extreme case of one-shot learning. Therefore, meta-learning algorithms must be able to effectively adapt to these uncertainties [26,27]. We approach this issue from optimization point and propose to use methods from the family of Simultaneous Perturbation Stochastic Approximation (SPSA) algorithms as a multi-task weights optimizer due to their robustness and successful application in various machine learning and control problems with uncertainties [28,22,29].
Gradient-Based Methods. As stated earlier, one of the proposed gradient-based approaches is to incorporate multitask weights in a computational graph of meta-learning algorithm (MAML and ProtoNet in this work). In that case Sample N C random classes 3: Sample random elements in S ti and Q ti 4: Compute task loss function L θ,ti (Q ti ) 5: end for ) via (4) 7: Use multi-task weights optimizer to update ω t 8: Update parameters θ t via SGD by L M T ξt backpropagation procedure is used to compute weights updates and a gradient-based optimizer is shared with the whole model. This multi-task modification of meta-learning is denoted by MTM Backprop. We also explored first-order optimization approaches as a separate multi-task weights optimizer described in Algorithm 1. Adam optimization method [30] was selected based on the results of experiments and due to multiple mentions in the deep learning literature. This approach is named MTM Inner First-Order by analogy with inner loop optimization step in MAML (1).

SPSA for Tracking method
When solving a meta-learning problem, algorithms face model uncertainties associated with a critical issue of limited amount of training samples per task, which may result in overfitting [25]. This phenomenon is especially pronounced in the extreme case of one-shot learning. Therefore, meta-learning algorithms must be able to effectively adapt to these uncertainties [26,27]. We approach this issue from optimization point and propose to use methods from the family of SPSA algorithms as a multi-task weights optimizer due to their robustness and successful application in various machine learning and control problems with uncertainties [22]. In this subsection, we propose the SPSA for Tracking algorithm firstly adapted for multi-task weights optimization.
In order to cast the SPSA-based approach as a multi-task weights optimizer, we reformulate the problem of finding estimates of hyperparameters ω t in the multi-task loss function (4) as a non-stationary optimization problem suitable for a stochastic approximation [31,4]. In order to do this, we introduce an observation model for the training episode ξ t that takes into account uncertainties arising in meta-learning: where ν t is an additive external noise caused by uncertainties arising from the limited amount of training samples per task. Therefore, it is necessary to find an algorithm that produces an estimate ω t of an unknown vector ω t that minimizes mean-risk functional of objective functions (5) based on observations L 1 , L 2 , . . . , L t from training episodes ξ 1 , ξ 2 , . . . , ξ t .
To the best of our knowledge, we propose to use SPSA for Tracking approach [31] as a multi-task weights optimizer for the first time. This algorithm simultaneously uses observation from the current and previous iterations which allows it to be more stable under parameters drift conditions. Let ∆ t ∈ R d be a vector consisting of independent random variables with Bernoulli distribution, ω 0 a vector with the initial values, and {α t } and {β t } sequences of positive numbers. Then SPSA for tracking multi-task weights optimizer (MTM SPSA-Track) constructs the following estimates of multi-task weights: Algorithm (6) has theoretical convergence guarantees expressed in terms of an upper bound of residuals between the estimates and the theoretical optimal solution [31].

Multi-Task Modification for Class-Based Weights
In Algorithm 1, N C classes in task t i is sampled randomly. However, if additional meta-information about classes is available, the data can be sampled from some coarse classes. For this, we define the modified multi-task loss function (4) applicable to coarse classes as well as to usual finer-grained classes. A per-task loss function L θ,ti can be rewritten as a sum of terms corresponding to classes: where k is a class, k ∈ {1, . . . , K}, K is the number of coarse classes (or K = C), L θ,ti,k (Q ti ) is the algorithmspecific partial loss computed for the corresponding class which is equal to zero if the class k is not present in the task t i . Then we modify multi-task loss function (4) by introducing weights ω (k) C for each class and combining like terms from M training tasks: SPSA-based multi-task weights optimizer with loss function (7) is denoted as MTM SPSA-Coarse.

Experiments
We conducted experiments on four datasets -CIFAR-FS [8], FC100 [13], miniImageNet [6] and tieredImageNet [32]. These datasets were selected as they became standard benchmarks over the last couple of years [15,7]. Experiments were performed on Rolos platform * with a A100 vGPU, 48 vCPUs at 2.30GHz, 128GiB RAM. Tables 1, 2   As mentioned in Section 2, the experiments with Multi-Task Modification were performed using MAML and Prototypical Networks. For MAML algorithm we used the neural network φ θ defined in the original paper [1] with 32-32-32-32 configuration where a-b-c-d denotes a 4-layer convolutional neural network with a, b, c, d filters in convolutional layers. Adam was used as the meta-optimizer with learning rate β = 10 −3 and adaptation step size α = 0.01 as in (2). We used 5 adaptation steps during training and 10 during testing for all datasets. We selected meta-batch size of M = 4 and trained the model for 300 epochs, each containing 100 tasks, unless specified otherwise. Following [33] and the original paper, 15 samples per class were taken for evaluation. For the MTM the baseline model was trained for extra 40 epochs. * https://rolos.com/ In the case of ProtoNet we used 64-64-64-64 feature extraction backbone φ θ as suggested in the original paper [5]. We followed the standard practice [34] for the meta-learning setup using SGD with Nesterov momentum of 0.9 and weight decay of 0.0005 as an optimizer. The learning rate was initially set to 0.1 and then decreased according to the strategy from [34]. During meta-training, we used the first 20 epochs for pre-training the model by using the original ProtoNet method and implementing the Multi-Task Modifications only for the last 40 epochs.
We designed several experiment settings to research the relative advantage of using the multi-task loss function (4) and SPSA-based optimization over original methods (Table 1) and over gradient-based method ( Table 2) where multi-task weights in the loss function are optimized jointly with the network parameters θ. In experiments we mainly used M = 4 tasks per training episode ξ t (other values of M are indicated explicitly) and the best model for testing was selected on the validation set. For SPSA and SPSA-Track we set α n = 0.25/n 1 6 , β n = 15/n 1 24 in (6) as per the theoretical result from [22]. During the experiments we found that L 2 -normalization of multi-task weights in MTM SPSA and MTM SPSA-Track improves the stability of training. Results of all experiments are formulated in terms of average few-shot classification accuracy after 1000 testing iterations. We used the miniImageNet dataset to compare our method with the prior work. Since the majority of recent approaches use more advanced convolutional neural networks with higher embedding dimension such as residual networks (ResNet) [24] as feature extraction backbones, we implemented the original ProtoNet with ResNet-12 backbone provided in [2] to compare against the results of other methods with backbones from the ResNet family. We did not include approaches that were developed for semi-supervised and transductive learning settings in this comparison since such approaches use the statistics of query examples or statistics across the one-shot tasks. We also excluded methods that use non-episodic pre-training as mentioned in Section 2. Table 3 shows that, when used with a comparable backbone, our multi-task meta-learning modification of ProtoNet with stochastic approximation increases one-shot classification accuracy to the level of significantly more advanced methods and is competitive against state-of-the-art meta-learning approaches. These results are presented with 95 % confidence intervals.  Table 1 show that proposed MTM with SPSA and SPSA-Track methods outperform the original approaches on all four benchmarks. Novel SPSA-Track demonstrates the largest improvement over the baseline in most cases.

Gradient-Based vs SPSA-Based Optimization.
We explored the comparison between gradient-based and SPSA-based approaches of multi-task weight optimization. As can be seen from Table 2, SPSA-based approaches give superior results in all experiments, so we can conclude that zero-order optimizers are better suited for the proposed multi-task modification in one-shot setting.

SPSA Multi-Task Weights Behavior
In MTM SPSA and MTM SPSA-Track experiments multi-task weights are renormalized during the optimization procedure after each epoch. The corresponding dynamics of SPSA-Track multi-task weights during training in miniImageNet 1-shot 2-way experiment is depicted in Figure 2 (a) for MAML and (b) for ProtoNet. Weights in ProtoNet experiment demonstrate larger fluctuation, yet in both cases there is a visible trend of multi-task weights splitting despite the difference in meta-learning methods.

Task Data Sampling
We used data sampling provided by Torchmeta library [42] to supply a unified approach to the problem. It follows the usual procedure where classes are sampled from the set of candidates and then an appropriate number of examples are selected per class. This procedure is randomized for every task and is independent from the choice of algorithm or its multi-task modification.
In MTM SPSA-Coarse experiment setting the tasks are sampled in the same way, however, information about coarse classes is passed to the MTM SPSA algorithm. We have modified MTM SPSA, so that instead of having M = 4 multi-task weights (one per task), we use 20 (one per CIFAR-100 coarse class). Consequently, instead of weighting tasks, we now weight instances of the corresponding coarse classes. A tie between a particular coarse class and the corresponding weight is established via modified multi-task loss function (7). At the end of a training episode the weights are updated only for the coarse classes that are present in the episode's meta-batch.
As can be seen from Figure 3 (a), in case of MAML the coarse classes weights ω (k) C dynamic during training varies significantly between different classes, which contrasts with ProtoNet, where weights follow the general dynamics established in [4] as shown in Figure 3

Conclusion
In this paper we have developed a new multi-task meta-learning modification with stochastic approximation for one-shot learning. The application of this approach to optimization-based method MAML and metric-based method ProtoNet was investigated on four most widely used one-shot learning benchmarks. In all experiments our algorithm showed significant improvements over baseline methods. In addition, in most cases SPSA-based approach was better than gradient method. We presented novel SPSA for Tracking algorithm as a multi-task weights optimizer which has demonstrated the largest performance boost on average. For future work, we aim to apply the described approach to state-of-the-art few-shot learning algorithms and to reinforcement learning.