{'fraction_non_alphanumeric': 0.041116854849680314, 'fraction_numerical': 0.018943000952251394, 'mean_word_length': 4.440333024976873, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 1, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Model-Agnostic Meta-Learning using Runge-Kutta Methods

Meta-learning has emerged as an important framework for learning new tasks from just a few examples. The success of any meta-learning model depends on (i) its fast adaptation to new tasks, as well as (ii) having a shared representation across similar tasks. Here we extend the model-agnostic meta-learning (MAML) framework introduced by Finn et al. (2017) to achieve improved performance by analyzing the temporal dynamics of the optimization procedure via the Runge-Kutta method. This method enables us to gain fine-grained control over the optimization and helps us achieve both the adaptation and representation goals across tasks. By leveraging this refined control, we demonstrate that there are multiple principled ways to update MAML and show that the classic MAML optimization is simply a special case of second-order Runge-Kutta method that mainly focuses on fast-adaptation. Experiments on benchmark classification, regression and reinforcement learning tasks show that this refined control helps attain improved results.


Introduction
Building an intelligent system that can learn quickly on a new task with few examples or few experiences is one of the central goals of machine learning. Achieving this goal requires an agent that learns continuously while having the ability to adapt to new tasks with limited data. Metalearning (Biggs, 1985) has emerged as a compelling framework that strives to attain this challenging goal.
There are two main approaches to meta-learning: learning-to-optimize and learning-to-initialize the meta-model (usually encoded as deep network). Learning-to-optimize refers to having a model that encodes the learning algorithm and predicts the direction of the parameter updates (Hochreiter et al., 2001). Learning-to-initialize refers to learning a representation that can quickly adapts to solve multiple tasks (Vinyals et al., 2016;Ravi and Larochelle, 2017;Finn et al., 2017). Here we focus on understanding and improving the latter approach, which is found a wide range of applications (Li et al., 2017;Nichol et al., 2018;Antoniou et al., 2019).
As one expects, over-fitting makes learning any new task from a few examples very difficult. Meta-learning overcomes the problem of scarcity by jointly learning from a collection of related tasks, each task corresponds to a learning problem on a different (but related) dataset.
Model-Agnostic Meta-Learning (MAML) has emerged as a popular state-of-the-art model in this framework (Finn et al., 2017). It is a gradient based optimization model that learns the metaparameters (that help to generalize to new tasks) in two update phases: fast adaptation (innerupdates) and meta-updates (outer-updates). Roughly, the inner-updates optimize the parameters to maximize the performance on new tasks using few examples, and the outer-updates optimize the meta-parameters to find an effective initialization within the few parameter updates.
Finding a model that can yield good prediction accuracy within a few updates requires (i) fast-adaptation -that is, finding model parameters that can either quickly change the internal representation (so as to maximize the sensitivity of the loss function of the new task), and/or (ii) * imd@janelia.hhmi.org shared-representation -that is, developing a high quality joint latent feature representations (so as to maximize the mutual information between different tasks). Motivated by this, we propose new learning-dynamics for MAML optimization that gives better flexibility and improves the model on both these fronts. Specifically, we apply the class of Runge-Kutta method to MAML optimization, which can take advantage of computing the gradients multiple-steps ahead when updating the meta-model. This allows us to generalize MAML to using higher-order gradients. Furthermore, we show that the current update rule of MAML corresponds to a specific type of second-order explicit Runge-Kutta method called the midpoint method (Hairer et al., 1987).
The main contribution of this work is as follows.
(i) We propose a novel Runge-Kutta method for MAML optimization. This new viewpoint is advantageous as it helps get a more refined control over the optimization. (Section 3.1) (ii) By leveraging this refined control, we demonstrate that there are multiple principled ways to update MAML and show that original MAML update rule corresponds to one of the class of second-order Runge-Kutta methods. (Section 3.2) (iii) The Runge-Kutta framework helps understand the MAML learning dynamics from the lens of explicit ODE integrators. To the best of our knowledge, this is the first work that successfully applies ODE solvers to meta-learning. (Section 3) (iv) We show that the refinement obtained by the Runge-Kutta method is empirically effective as well. We obtain significant improvements in performance benchmark classification, regression, and reinforcement learning tasks. (Section 5) 2 Model-agnostic Meta-learning (MAML) Model-Agnostic Meta-Learning (MAML) was introduced by Finn et al. (2017) with the aim to train a model that can adapt to a large set of new tasks with only a few data points in a few learning iterations. Given a meta-model f parameterized by meta-parameters θ, one wants to find a gradient learning rule can make rapid progress on a new task. This can be formalized as follows: for a task T i drawn from a distribution of tasks p(T ), we require that small changes in parameters using gradient based parameter updates, that is (for any given learning rate h) would result in large improvement on loss function L of a task T i . This requirement implies that meta-parameter θ is an initialization that produces a more transferable internal representation and achieves good prediction with only a few examples from a new task. MAML finds such a meta-parameter by simultaneously minimizing loss functions associated with each task. The MAML meta-objective is defined as (Finn et al., 2017) where θ is as per Eq. (1), and the total loss L(f (θ )) is simply the aggregate of the individual task specific losses Ti∼p(T ) L Ti (f (θ )). Thus, the meta-parameter gets updated by taking a gradient descent step in the direction that minimizes loss for all the given tasks. Algorithm 1 provides an overview of the MAML training procedure. A batch of tasks is sampled from the task distribution p(T ). The model parameters are then updated for each task (innerupdates). These updated parameters are then used to update the meta-parameter (outer-update).
For simplicity and ease of subsequent discussion, we will hide the model f inside the loss L and refer to it as L(θ ) henceforth. That is, L(θ ) := L Ti (θ − h∇ θ L Ti (θ)).

Generalizing MAML using the Runge-Kutta method
One of the central objectives of this paper is understanding the learning dynamics of MAML through the lens of explicit ODE integrators. Consider the vector field X that maps the space of for all T i do

3:
Sample batch of tasks T i ∼ p(T )

4:
for all T i do

5:
Evaluate ∇ θ L Ti 6: Update model parameter: Update meta-parameter: parameters to the descent directions induced by the gradient of the MAML objective. So long as X is sufficiently smooth, we can look for solutions of the form The temporal dynamics of this ODE constitutes the ideal path that the meta-learner takes during training given an initial parameter θ 0 . We can therefore use a numerical ODE solver to solve approximate this ideal solution. Clearly a higher order explicit ODE integrator can be viewed as a black box that transforms X into a new vector fieldX such that the parameters θ evolve more closely to the streamlines of X . For gradient descent, X = ∇ θ L and the resultingX advects θ more closely along the path of the steepest descent. One of the central goals of this work is to precisely to discern the efficacy of applying integrators onX rather than X . To do this we will replace the gradient X (θ t ) = ∇ θ L(θ t ) with the appropriateX (θ t ) by calling a chosen explicit ODE integrator. This viewpoint generalizes the MAML optimization framework to optimize with respect to temporal parameters in contrast to considering the spatial parameters as done in the previous literature (Park and Oliva, 2019;Chen et al., 2018;Song et al., 2019). For a given timestep t, explicit integrator can be seen as a morphism over vector fields X →X h (for a fixed stepsize h). Hence, for a true gradient g t = ∇ θ L(θ t ) (at time t), we solve the modified Runge-Kutta gradientḡ t = ∇ θ L(θ t ) as follows. Define The general form of advect RK gt (θ, h) is the Runge-Kutta (RK) equation of order N (Butcher, 2008), given by where (i) a i are combination weights (that should sum to 1), (ii) p j are the so called nodes that scale the timestep (for better numerical approximation), (iii) q ij are the coefficients that scale the step towards the gradient k i (for a fine grain optimization control), and (iv) i−1 j=1 q ij = p j for all j = 2, . . . , N . The specific choice of these parameters (a i , p j and q ij ) gives rise to various popular instantiations of the RK optimization method.  Figure 2: Diagram of MAML-RK illustrates how parameters θ evolve over time. The higher-order MAML-RK takes the path that is closer to continuous path X (bold red curve) that is closer to best parameter initialization that can quickly adapts to multiple tasks. Figure 1 demonstrates the slopes k i for a quadratic function (depicted in blue color) for 1 ≤ i ≤ N = 4. Notice that it takes a linear combination of the k i 's (red arrows) to calculate the next step vector (green arrow). These k i 's can be thought as forward multi-steps since they compute the gradients at future timesteps (c.f. Figure 1). Rearranging the terms with respect toḡ t , we get We can use this refined Runge-Kutta gradient in the MAML optimization for better convergence and will call it as the generalized Runge-Kutta MAML (MAML-RK) method. Observe that g MAML-RK t is a linear combination of the gradients of the forward multi-steps k i over the sum of individual task specific loss functions L Ti (t, θ t ). Unlike the standard MAML update, which takes the gradient only one step forward (inner-update) 1 , MAML-RK has the ability to take the gradient multiple steps ahead. This encourages the meta-learner to find parameters that adapt to new tasks with very few (sometimes even just one) gradient steps on a new task, just as what we desire. Note that although we advect g t with RK in this work, any other other explicit ODE solver can also be used. Figure 2 illustrates of the optimization path of MAML-RK. The bold blue path represents the continuous learning dynamics of the meta-parameter X (θ). The three parameters θ 1 , θ 2 , and θ 3 are the optimal parameters for three tasks, and the best meta-parameter θ * is chosen in such a way that it lies close to all of them. The path of standard MAML optimization (the green curve) deviates from X (θ) due local crude approximations. Using higher-order explicit RK methods (the red curve), follows the ideal path better due to better quality approximation.
Next we show that the original MAML optimization is a special case of MAML-RK (with a specific setting of a i , p j , q ij ) and also explore a wider setting of these parameters giving us other types of first and second-order optimizations for MAML.

Advect MAML's gradient
The technical key component of MAML is that it computes the gradient with respect to the meta-parameter θ while computing the objective on the updated model parameter θ (c.f. Equation 2 and Algorithm 1). This makes the meta-parameter to move towards the direction of ∇ θ Ti∼p(T ) L Ti (θ i ). Observe that this precisely corresponds to the k 2 component of MAML-RK with a specific setting of p 2 and q 21 . The following proposition tells us that MAML is a special case of second-order MAML-RK. Proposition 1. The MAML's gradient corresponds to the second-order explicit Runge-Kutta equation with the parameters a 1 = 0, a 2 = 1, q 21 = 1 2 , p 2 = 1 2 . Proof. For simplicity, let us denote ∇ θ L(t, θ t ) as ∇L(t, θ t ). Consider the Taylor expansion of MAML's gradient: We can compare Equation 5 with the second-order explicit Runge-Kutta Equation for MAML, which is such that a 1 + a 2 = 1, a 2 q 21 = 1 2 , a 2 p 1 = 1 2 . We can now see that Equation 6 equals Equation 5 with a 1 = 0, a 2 = 1, p 2 = 1 2 , and q 21 = 1 2 . The optimization done with the specific setting of the RK parameters as shown in Proposition 1 is usually called the midpoint method (Butcher, 2008). This shows that the original MAML objective is essentially a midpoint optimization method.
A setting of a 1 = 0 and a 2 = 1 implies that the classic MAML objective solely relies on ∇L θ (t + p 2 h, x t + q 21 k 1 h). Moreover, a setting of q 21 = 1 2 corresponds to a learning rate of innerupdate (i.e. α in Algorithm 1) to be 1 2 h and the learning rate of outer meta-update (i.e. β in Algorithm 1) to be h. This results in a meta-model optimization to stays as close to the ideal trajectory path X (θ) with an error rate of O(h 3 ) per step (c.f. proof of Proposition 1).

Examples beyond MAML
The Runge-Kutta gradientḡ MAML-RK t discussed so far has been generic. One can instantiate it (i) at various degrees of order (by choosing N ), and (ii) by varying the RK parameters in Equation 4. In this section, we examine some example instantiations of MAML-RK.
Example 1 (First-order MAML). Choose N = 1 and a 1 = 1. Then,  x The first-order MAML is simply the Euler's method. It sums over the gradients of the loss functions for every task. It is similar in flavor to other first-order approaches, such as FO-MAML (Biswas and Agrawal, 2018) and Reptile (Nichol et al., 2018).
In the previous section, we showed that MAML is a special case of second-order MAML-RK by advecting g t with MAML's gradient. Here, we illustrate several other popular second-order Runge-Kutta methods that we can also apply to MAML gradients, thus extending the variety of optimization techniques that are currently in use for meta-learning.
We can thus generalize Algorithm 1 by substituting the learning rate α as q 21 h, and β as h. Note that q 21 = p 2 for all second-order Runge-Kutta methods. Algorithm 2 presents generalized MAML-RK2 training algorithm. It is worth noting that the existing literature only discusses the midpoint method for training MAML, and our extension enables the practitioner to explore other methods, like Heun's, Ralston, and ITB Hairer et al. (1987).

Additional Remarks
Using the gradient (ḡ MAML-RK1 = ∇L(θ)) from a pre-trained model that uses a large dataset and then fine-tuning it on a smaller new dataset is popular in transfer learning and has become a popular techniques in various application domains. For example in computer vision, it is common to use a parameters and gradients of a pre-trained network on ImageNet (Deng et al., 2009) and use it to perform, say, bird classification (Zhang et al., 2014). Hence, there is some evidence that pre-training with first order gradient, i.e.ḡ MAML-RK1 helps the model learn a shared feature representation that can be applied across similar tasks. Although it is worth noting that it does not encourage the model to learn a meta-parameter that can rapidly adapt to a new task. In contrast, part of the success of the classic MAML optimizationḡ MAML (a specific second order method) is because it can directly optimize for this rapid new task adaptation by differentiating through the fine-tuning process with respect to the meta-parameter ∇L(θ ), but it does not make use of the first order gradient ∇L(θ). it is important to note our second order Runge-Kutta generalization g MAML-RK2 (with specific instantiations as Heun's, Ralston, and ITB), considers both the terms: ∇L(θ) and ∇L(θ ), and thus have the potential benefit of encouraging both rapid adaptation and shared feature representation (Raghu et al., 2019). See our experiments in Section 5.
For every meta-learning update, the first-order method performs one evaluation of L, and the second-order method performs two evaluations for RK methods. The number of evaluations grows linearly up to the fourth-order, after which it grows faster making it computationally prohibitive. The fourth-order Runge-Kutta method is often the popular method for solving initial value ODE problems. In our case, even the second-order optimization requires Hessian-vector products during the MAML updates and more evaluations is impractical. We will therefore limit to second order RK methods in our experiments.

Related Work
Early approaches to meta-learning goes back to the late 1980s and early 1990s (Schmidhuber et al., 1987;Bengio et al., 1991) where it studies evolutionary principles in self-referential learning using genetic programming. There has been a recent surge in interest where meta-models are applied to a wide array of tasks from architecture search (Zoph and Le, 2016) and hyperparameter search (Maclaurin et al., 2015), to learning optimization . It has become a popular approach in few-shot supervised learning (Hariharan and Girshick, 2016) and fast reinforcement learning (Wang et al., 2017).
Among many approaches to meta-learning, Finn et al. (2017) proposed the MAML framework that uses a meta-objective function that performs a two-step gradient-based optimization of the meta-parameters for fast task-specific optimization. Various studies (Li et al., 2017;Antoniou et al., 2019) show that a fast-adaptation update rule can heavily influence performance, which has initiated several related investigations. Biswas and Agrawal (2018) and Nichol et al. (2018) use first-order update methods to reduce computational burden, while Park and Oliva (2019),    Chen et al. (2018) and Song et al. (2019) leverage second-order curvature information for better generalization. Instead of analyzing the spatial features (such as gradient and curvature of the parameter space), our work focuses on the temporal dynamics of the optimization. Recent work by Raghu et al. (2019) analyzes the source of effectiveness of MAML -is it primary due to rapid learning, or due to feature reuse? Their analysis shows MAML partly achieves both -large portion of the lower layers of the MAML model helps in feature reuse and large portion of the upper layers helps in rapid learning. Part of the goal of our work is to think about aspects of a meta-model more effectively. Our RK extension makes fast-adaptation and shared-representation more explicit giving practitioners a fine grain control over the optimization.

Experiments
To study the effectiveness of our extended Runge-Kutta MAML framework, we conducted detailed empirical studies of various explicit instantiations of second order MAML-RK methods (as detailed in in Section 3) on various classification-, regression-and reinforcement-meta-learning benchmarks. Throughout the experiments, we compare the midpoint (i.e. the original MAML optimization), Heun's, Ralston, ITB methods. The data, models, and the optimizer for all our experiments is built upon the original MAML code 3 .
The following standard setup is used in all our experiments. All models are trained from the training task dataset and evaluated on the test task dataset. For each task, we have a support set of K examples, which are used for fast-adaption updates. During the evaluation phase, the model is initialized with the learned meta-parameters from training phase, and is fine-tuned on the K samples from the test tasks. The model architecture for each experiment can be found in the Appendix.
Regression -Following the experiments in MAML (Finn et al., 2017), we consider the sinusoid regression problem. For each task, the 1-dimensional sinusoid wave of amplitude and phase are varied between [0.1, 5.0] and [0, π], and the goal is to regress on an unseen sinusoid wave. The datapoints are sampled from the range of [−5.0, 5.0] and we used batch size of ten (K = 10) for every gradient update with a fixed step size of 0.01. The mean-square-error is used as a loss  Table 2 presents the regression performance. The MAML-RK1 (the first-order method) corresponds to the pre-trained model on all training tasks, which is the baseline method. Note that MAML-RK2 (midpoint) corresponds to original MAML method. We observe that MAML-RK2 (Ralston) performs marginally better than the midpoint method for this simple task.
Classification -Next, we evaluate MAML-RK on classification tasks. It has been shown that MAML (Finn et al., 2017) achieved state-of-the-art performance when compared to prior metalearning and few-shot learning algorithms (Koch, 2015;Vinyals et al., 2016;Ravi and Larochelle, 2017) on few-shot Omniglot (Lake et al., 2011) and MiniImagenet (Ravi and Larochelle, 2017) image recognition tasks. Therefore, we only report MAML's results against other extended MAML-RK2 methods. The standard setup for few-shot classification is that we consider 5 and 20 classes (N -way) with 1-shot and 5-shot learning, and evaluate on new 5 and 20 classes.
The Omniglot images were downsampled to 28 × 28 and were augmented with up to 90 degrees of rotation. The training task classes were randomly selected from 1200 out of 1623 characters and rest were used as test task classes. We followed the same model architectures from previous studies (Vinyals et al., 2016;Finn et al., 2017). For MiniImage classification, the dataset consists of 84×84 60,000 colored images. There are 100 different classes where each class consists of 600 images. Out of the 100 classes, the training, validation, and test classes were split as 64, 12, and 24 respectively. Table 3 presents the performance on Omniglot dataset for 5-way and 20-way classification for 1-shot and 5-shot learning. For 5-way 1-shot and 20-way 1-shot learning, we observe that Ralston method performs the best, and followed by Heun's and ITB. The midpoint method performs the worst. We suspect that this is because only the midpoint method has zero coefficient for a 1 , which means that it cannot take the first-order gradient information ∇L(θ) into account. Hence, setting the coefficient a 1 to be greater than 0 is important for achieving better results on this dataset. Again for 5-way 5-shot learning, we observe that Ralston method performs the best and the midpoint method performs the second best. Figure 3 (a-c) illustrates that Ralston outperforms MAML throughout the training process. For 20-way 5-shot learning, we found that all MAML-RK2 models perform more or less the same, where the range of mean plus and minus the standard deviation overlaps. Heun's method performed the best for the MiniImagenet dataset. Reinforcement Learning -We evaluate MAML-RK2 on two types of reinforcement learning environments, 2D navigation and locomotion. For training, REINFORCE was used for policy (Williams, 1992) and trustregion policy optimization (TRPO) was used for meta-optimization (Schulman et al., 2015). For 2D navigation, there are a set of tasks where a point agent must reach to different goal locations in 2D space. The state, the action, and the reward corresponds to the 2D location, the motion velocity, and the distance to the goal respectively. The simulation terminates when an agent navigates within 0.01 distance from the goal. Figure 5 presents the performance of MAML-RK2 for up to nine gradient updates with 40 samples. A model trained with random initialization (black triangle line) performs poorly even with nine gradient updates. The red curve corresponds to the performance of model with oracle policy that receives the goal position as input. Recall that a 2 corresponds to fast adaptation coefficient and a 1 corresponds to good feature representation (because the higher the coefficient, the more it focuses on ∇L(θ ) and less on ∇L(θ)). Interestingly, according to the plots in Figure 5 the learning speed is ordered as follows: the midpoint, Ralston, Heun's, and ITB methods. This ordering corresponds to the ordering of coefficient of a 2 , which is 1, 2 3 , 1 2 , and 1 3 respectively. The midpoint method suffers from poor performance the most. We suspect that this is because it only emphasizes ∇L(θ ) (fast-adaptation) part and ignores ∇L(θ) (sharedrepresentation) part. Ralston method seems to achieve the right balance between the two terms as the performance reaches to the oracle (red level) the fastest.
Lastly, Figure 4 illustrates the actual trajectory of learning towards the final location. The results shows that the midpoint method takes long time to find the goal location and jitters a lot (showing suboptimal temporal dynamics). On the other hand, ITB finds the goal location within very few steps. Although Ralston and Heun's are in between the midpoint and ITB, both methods still take much fewer steps to converge to the goal. This clearly demonstrates the role between optimizing under ∇L(θ ) versus ∇L(θ).

Discussion and future work
In this paper, we extend the fast-adaptation stage (the inner loop) to higher-order Runge-Kutta methods in MAML to gain a finer control over the optimization, and show that original fastadaptation update corresponds to the second-order midpoint method. The refined RK optimization helped us control various important aspects of the meta-learning process (fast adaptation and shared representation) achieving improved performance on regression, classification, and reinforcement learning tasks.
It is worth noting that our proposed generalization is not specific to MAML, but can also be applied to other meta-models. We share some potential directions of future work.
Exploring other ODE integrators -We applied explicit Runge-Kutta ODE integrator to generalize stochastic gradient optimization of MAML. One can also explore other variations on gradient-based updates such as AdaGrad and ADAM (Duchi et al., 2011;Kingma and Ba, 2015) and its effects to the meta-learning models. (Similar types of analysis has been done for image classification via neural networks, Im et al., 2017.) Beside RK integrators, one can also apply other integrators, such as exponential-and leapfrog integrators. Since different integrators focus on different aspects of the optimization, one expects that they would benefit on different types of tasks. We believe that a thorough analysis of this would be an interesting direction to explore in the future and would be extremely beneficial to the practitioner.
Extension to ANIL - Raghu et al. (2019) recently showed that feature reuse is the dominant factor in MAML optimization and propose to only train adaption updates on the last layer (i.e. the softmax layer for classification) of the model. It would be instructive to study how different RK methods effects the various layers of the MAML network. For example, we can use the midpoint method for the last layer and apply Ralston, Heuns, ITB, and the gradient for the lower layers. This essentially has the effect of shifting the balance from a 2 to a 1 as we move down to the lower layers.
Extension to Bayesian MAML -There are several works on Bayesian MAML (Kim et al., 2018;Finn et al., 2018) that help in adding robustness and preventing overfitting to few shot learning. It would be interesting to combine MAML-RK optimization with these frameworks.

A Experimental Details
Here, we include some of the experimental set-up details. Note that we worked based on original MAML code base. This makes the the experimental set-up the same as the original paper.

A.1 Regression
We used multilayer perceptrons with two ReLU hidden layers of size 40. We trained each model fore 100,000 iterations. Every gradient updates are using 10 batch samples. The learning rate (step size) is set to 0.01.

A.2 Classification
Omniglot dataset Following the same model architectures from previous studies (Vinyals et al., 2016;Finn et al., 2017), we used four convolutonal blocks with 64 3 × 3 convolution filters, followed by batch normalization and ReLU activations, and 2 × 2 max pooling. The last hidden layer is 64.
We ran 30,000 epoch for 5-way 1-shot and 5-way 5-shot set-ups for all models. We ran 50,000 epoch for 20-way 1-shot set-up for all models. We ran 90,000 epoch for 20-way 5-shot set-up for all models.
Because there are total of N K examples, where N is number of tasks and K is number of examples per task, we were able to use 32 batch sizes for each gradient updates for meta-parameters. The network was evaluated using 3 gradient updates with step size 0.4 for 5-way set-up and 5 gradient updates with step size 0.1 for 20-way set-up.
MiniImagenet dataset The same convolutional blocks are used as Omniglot dataset except that the number of filters were reduce to 32. The batch size was set to 4 and 2 for meta-parameter updates. All models were trained for 90,000 epoch and 5 gradient updates with 0.01 learning rate during the training. At test time, 10 gradient updates were used for 15 examples per class.

A.3 Reinforcement learning (2D navigation)
For all meta algorithms presented in this paper, the parameters are set to be the same. In the 2D navigation, like the original MAML paper, the model consists of a 2-layer MLP each with 100 units and it uses ReLU for activation. The fast adaptation learning is rate is 0.16 while the meta learning rate is 0.01. We trained it with 100 iterations and meta batch size of 40.