{'fraction_non_alphanumeric': 0.0310934393638171, 'fraction_numerical': 0.011530815109343936, 'mean_word_length': 4.716136363636363, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Learning Rapid-Temporal Adaptations

A hallmark of human intelligence and cognition is its flexibility. One of the long-standing goals in AI research is to replicate this flexibility in a learning machine. In this work we describe a mechanism by which artificial neural networks can learn rapid-temporal adaptation - the ability to adapt quickly to new environments or tasks - that we call adaptive neurons. Adaptive neurons modify their activations with task-specific values retrieved from a working memory. On standard metalearning and few-shot learning benchmarks in both vision and language domains, models augmented with adaptive neurons achieve state-of-the-art results.


Introduction
The ability to adapt our behavior rapidly in response to external or internal feedback is a primary ingredient of human intelligence. This cognitive flexibility is commonly ascribed to prefrontal cortex (PFC) and working memory in the brain. Neuroscientific evidence suggests that PFC uses incoming information to support task-specific temporal adaptation and planning [38,35,22,5]. This occurs on the fly, within only a few hundred milliseconds, and supports a wide variety of task-specific behaviors [25,31].
On the other hand, most existing "AI" systems are designed for a single task. They are trained through one long optimization phase after which learning ceases. Systems built in such a train-andthen-test manner do not scale to complex, realistic environments and tasks: They require gluts of single task data and are prone to issues related to distributional shifts, such as catastrophic forgetting [37,8,16] and adversarial data points [39].
There is growing interest and progress in building flexible, adaptive models, particularly within the framework of metalearning, or learning to learn [24,1,42,2]. Previous work on metalearning has formulated the problem as two-level learning; specifically, "slow" learning of a meta-level model that occurs across several tasks, and "fast" learning of a base-level model that acts within each task [24,41,29,7,26,23]. The goal of the meta-level learner is to acquire generic, more abstract knowledge that applies broadly. This knowledge can then be transferred to the task-adapted base-level learner to provide generalization in the context of a single task. The base and meta-level models can be framed as a single learner [34] or as separate learners [3,14].
In this work we describe and evaluate a neural mechanism for rapid-temporal adaptation. Our proposed model uses adaptive neurons that modify their own activation values on the fly at any time. They achieve this activity modification as a function of meta information. Succinctly, meta information encodes the behavior or procedure of a model as it performs a task. It can be extracted from a model at training or inference time. In the metalearning setup, we can train a meta model to learn how to use meta information to adapt a base model to new tasks. We here define and investigate two forms of meta information, one based on error gradients and another based on direct feedback [21,27]; however, various other types of meta information are possible.
Neuron-level adaptation is highly efficient compared to previous methods that adapt synaptic connections between neurons, for instance via fast weights [26] or an optimizer [7,29]. Neural activity can be modified straightforwardly over different time-scales within separate task and objective contexts. Furthermore, since the concept of adaptive neurons is generic, they can be used to augment various neural architectures, including convolutional and recurrent networks. We demonstrate empirically that ResNet [9] and deep LSTM [13] networks equipped with adaptive neurons achieve 57.1% and 70.04% accuracy on the standard Mini-ImageNet 1-shot and 5-shot benchmarks and 57.25%, 68.85%, and 69.5% accuracy on Penn Treebank 1, 2, and 3-shot language modeling tasks. This marks a significant improvement over the previous state of the art.

The Framework for Rapid-Temporal Adaptation
The concept of meta information plays the main role in our framework for rapid-temporal adaptation (RTA). While a model or an agent M experiences a sequence of relevant or irrelevant tasks, we seek to extract information about the model itself and to adapt the model temporarily based upon it. Thus, the meta information I is a type of feature that describes the model in the context of a task or an environment.
We assume that for each task τ we have access to a set of task description inputs {x i } N i=1 ∈ D τ and unseen examples {x i } L i=1 ∈ τ coming from the same task; depending on the availability of the data, this description may be full or partial. For example, a full description could be input pairs of examples and labels {x i , y i } N i=1 in the supervised learning setup where goal is to map new inputs ∈ τ , or action, state and reward triplets {s i , a i , r i } N i=1 ∈ τ observed in an environment in the reinforcement learning setup where we learn a policy that maps new states When certain parts of the full description are missing (e.g., we only have access to input examples but not their labels), we call the description partial. Note that a partial description {x i } N i=1 still contains information about the input task distribution and this can be useful in learning adaptive generative models [30].
Concretely, let us assume the model M changes its state S to S after processing the description D τ of the current task τ .
We then define the meta information I S ,S that describes the model in the current task space as where f I is a function that extracts the meta information. The model temporarily adapts to the task τ by taking in its own meta information: We apply this adapted model M τ to solve task τ . The adaption just described occurs rapidly, within a limited resource regime. Furthermore, if the meta information I S ,S is general, meta-level knowledge transfer and continual learning are naturally supported within the framework.
As a learning framework, RTA relates to zero-or one-shot learning depending on the given task description. When a full description consisting of only one or a few examples per label is provided for each task, it resembles one-shot learning [42,40]. In the scenario where labels are missing for some input examples and yet additional side information is provided as the partial description, it closely resembles zero-shot learning [19]. In both cases, a model with abilities of temporal adaptation and knowledge transfer is crucial. Our RTA framework implicitly transfers meta-level knowledge across similar tasks through the meta information, since meta information extracted from the model performing a set of similar tasks is likely to encode more general, transferrable knowledge.
We evaluate two variants of first-order meta information: an error gradient and a "direct feedback" [21,27]. While the former requires a procedure for backward error propagation (i.e., backprop), the latter is cheap to obtain.
Error gradient as meta information We simply apply the chain rule and the standard backprogation algorithm to a given neural network M to obtain the loss gradient for each neuron as meta information. For a model with softmax output, let backprop be the error backpropagation procedure and y be the true label given the description. Then we obtain the loss gradients for neurons at layer t asŷ = softmax(a(T )), where a(T ) is the pre-activation of the output neurons andŷ the label prediction (i.e., the softmax output). We denote a loss function, such as the cross entropy loss, with L(·). Because backpropagation is inherently sequential, the meta information here is expensive to calculate. The process becomes expensive for very deep networks, such as RNNs processing long sequences. This motivates us to propose the direct feedback meta information.
Direct feedback meta information Direct feedback (DF) meta information is inspired by feedback alignment methods [21,27] and on biologically plausible deep learning [4]. We obtain the DF meta information for neurons at layer t as where σ (·) represents the derivative of the nonlinear activation function σ; on the right side, (ŷ −y ) is the derivative of the cross entropy loss with respect to the softmax input. |a| is the size of the neurons and j is the index for the neurons. Therefore, the DF meta information for each neuron at a particular layer is simply the derivative of the loss function scaled by the derivative of the activation function.
The set of meta information for all neurons defines the meta information for the entire network. We compute the DF meta information for the whole model simultaneously with a single multiplication, which is very efficient compared to backprop-based error gradients. To obtain the DF meta information it is sufficient that only the loss and neuron activation functions be differentiable. This is more relaxed than for backpropagation methods, which require every layer to be differentiable. This could be beneficial in models with non-differentiable components.
We demonstrate the effectiveness of both variants of meta information in Section 5.

Adaptive Neurons
The idea of adaptive neurons is to modify or bias a network's activations by shifting them according to extracted meta information. Adaptive neurons take the following form: for hidden layer t and output layer T , where σ is a nonlinear elementwise activation function and β t is the task-specific bias determined from the meta information. Like standard artificial neurons, a single adaptive neuron has an activation that depends on the preceding layers. In addition, it has a task-specific activation shift to improve performance in the current task context. The latter, after passing through the nonlinearity σ, provides a primitive update that can help to stabilizing the learning of succeeding layers. Since the task-specific activation varies from task to task and domain to domain, also it is directly applied to the activation itself, the adaptive neurons can regularize the weights of the following layers.  To implement a model with adaptive neurons, we must define functions that transform the meta information I S ,S into the activation shifts σ(β(t)). For this we modify the MetaNet architecture of Munkhdalai and Yu [26]. MetaNets consist of a base-learner plus a shared meta-learner with working memory. For each task τ , the model processes its full description {x i , y i } N i=1 ∈ D τ and stores relevant meta information in working memory. To classify unseen examples {x i , y i } L i=1 ∈ τ coming from the same task, the working memory is queried with examples {x i } L i=1 , using an attention mechanism, to generate a set of fast weights for the base-learner. The attention mechanism is implemented with a representation learning function (RLF). Here we replace the fast weights of MetaNets with the task-specific activations of adaptive neurons. We extract meta information from and for each neuron, in the form of the error gradient or the direct feedback.
Adaptive neurons can straightforwardly be incorporated into various deep neural architectures. We start by describing the model details for a feed-forward network, then show variants for the ResNet and LSTM architectures.

Feed-Forward Networks with Adaptive Neurons
For simplicity we use feed-forward neural networks (FFN) with ReLU activation for both the base learner and the representation learning function. Each network respectively features one hidden layer and one output layer, without loss of generality. The overall architecture of the model is shown in Figure 1.
The first step is to construct a task-specific representation for input data x , then use it to index the working memory. This is achieved through the representation learning function (RLF), which also provides input for extracting meta information, as follows: where a (1) RLF and a (2) RLF are pre-activation vectors corresponding to the input mini-batch x , and the matrices W and biases b are learnable parameters of the RLF. Function f I is defined as in eq. 5 or 6 and extracts the meta information for the neurons at each layer (i.e., the single hidden layer in this simple FFN). Next the meta information is transformed into the task-specific shift β RLF , which is used to adapt the RLF to the input task. The shift is computed as follows: where [·; ·] represents vector concatenation, f RLF is the meta weights, which is a deep feed-forward neural network, and max(), min(), avg() are pooling functions that operate batch-wise. This shifts the activations of the neurons in the RLF to produce key vectors K = {k i } N i=1 for the working memory: During training and test phases, working memory queries K = {k i } L i=1 are generated according to eq. 12 as well, given inputs {x i } L i=1 ∈ τ (instead of the task descriptor x ). The second step is to learn the task-specific shifts used to adapt the base-learner, and store them in the working memory indexed by K . Again we obtain the meta information I Base1 and I Base2 using the task description processed by the base-learner: The meta information is transformed into a pool of the task-specific shifts as follows: f base is again a deep feed-forward neural net that learns to map the inputs to memory contents. Matrices V 1 ∈ IR N ×h B 1 and V 2 ∈ IR N ×h B 2 are the values of working memory, in which N is task description size, h B1 and h B2 represent dimensionalities of hidden states in W B1 and W B2 , respectively. The memory rows in V 1 and V 2 represent stored activation shifts for the respective layers of the base learner. 1 For training and test inputs {x i } L i=1 ∈ τ , the working memory is read from using soft-attention in order to retrieve the task-specific shifts β base1 and β base2 : where norm is a normalization function (typically softmax). Finally we feed the inputs forward to obtain the task loss and optimize the model parameters end-to-end: where h (1) base and h (2) base are the shifted activations of the adaptive neurons. θ is the model parameters and optimize performs a gradient-based update on the parameters. We use the cross entropy loss for L.

Deep Residual Networks
For ResNets [9] we incorporate the adaptive neurons into the output of a residual block. Let us denote the residual block as ResBlock(), which is defined as follows: where x and a(out) are the inputs to the block and the output pre-activations, respectively. conv 3×3,d () or conv 1×1,d () denotes a 3 × 3 or 1 × 1 convolutional layer with d filters, batchNorm() represents a batch normalization [15] layer. Finally the activations h(out) of the adaptive neurons for ResBlock() is calculated as: where β a(out) is the task specific bias retrieved from the working memory that is constructed in the same way as the simple feed-forward network. The meta information here is calculated for the neurons at the output of each residual block. We stack several residual blocks with adaptive neurons to construct a deep ResNet model.

Long Short-Term Memory Networks
For LSTM [13] models the hidden state at each time step is biased towards the input task resulting in both space and time adaptation. Particularly, given the current input x t , the previous hidden states h t−1 , and the memory cell states c t , an LSTM model with adaptive neurons calculates its gates, new memory cell states and hidden states at the time step t by using the following update rules: where represents element-wise multiplication, β ct is the task specific bias coming from the memory V t that is constructed by processing the meta information of the memory cell, in the same way as the feed-forward network. By stacking such layers together we build a deep LSTM model that adapts in both space and time.

Related Work
Mitchell et al. [24] used the reward gradients from neural networks to better generalize to robot control tasks, whereas Schmidhuber [33] discussed the use of network weight matrices themselves for continuous adaptation of an agent in dynamic environments. Such additional information extracted from neural networks can be seen as meta information.
Among many different problems in supervised, reinforcement, and unsupervised learning that can be framed within the metalearning or rapid-temporal adaptation frameworks, one-shot learning has emerged as a natural and popular test bed. One-shot supervised learning refers to a scenario where a learner is introduced to a sequence of tasks, where each task entails multi-class classification given a single or very few labeled example per class. A key challenge in this setting is that the classes or concepts vary across the tasks; thus, a model requires a capacity for rapid adaptation in order to recognize new concepts on the fly.
As highlighted, the architecture of our model with adaptive neurons is closely related to Meta Networks (MetaNets) [26]. The MetaNet udpates synaptic connections (weights) between neurons using fast weights [34,12] to implement the rapid adaptation. While the MetaNet fast weights allow for flexibility, it is also very expensive to update these weights since the connections are dense. Neuron-level adaptation with adaptive neurons is significantly more efficient while lending itself to a range of network architectures including ResNet and LSTM. Indeed, MetaNet fast weights and adaptive neurons may complement each other, as the former provide adaptive connections between neurons and the latter provide adaption to the neuron activity itself.
As a form of feature-wise transformation method, the adaptive neurons are broadly related to conditional normalization techniques [20,6,28]. They proposed to modulate the pre-activations of neurons 2 whereas in adaptive neurons, we directly transform the neuronal activation itself in a similar way as the application of dropout [36] of neurons.

Experimental Evaluation
We evaluated the proposed approach on tasks from both vision and language domains. For the vision domain, we used two widely adopted one-shot classification benchmarks: the Omniglot and Mini-ImageNet datasets. The Omniglot dataset consists of images across 1623 classes with only 20 images per class, from 50 different alphabets [18]. Like previous studies, we randomly selected 1200 classes for training and 423 for testing and augmented the training set with 90, 180 and 270 degrees rotations. The images are resized to 28×28 pixels for computational efficiency. For the experiments on Mini-ImageNet data, we evaluated on the same class subset released by Ravi and Larochell [29]. This dataset has 84 × 84 color images and 100 classes (64/16/20 for training/validation/test splits) and each class has 600 images.
To evaluate the effectiveness of the LSTM model with adaptive neurons, we applied it to a one-shot PTB language modeling task introduced by Vinyals et al. [42]. Following [42] we used the same target words for the test data 3 and split the PTB sentences into training and test such that both target words and sentences in the test set are unseen for the model during training.

One-shot Image Classification
For the Omniglot benchmark we performed 5-and 20-way classification with one or five labeled examples provided as the full description for each task. We used a convnet with 64 filters as the base learner. This convnet has 5 convolutional layers, each of which uses 3 × 3 convolutions followed by a ReLU nonlinearity and a 2 × 2 max-pooling layer. Convolutional layers are followed by a fully connected (FC) layer and a softmax layer for output. Another CNN with the same architecture is used for the representation learning function. The last four layers of the CNN components are equipped with adaptive neurons (adaCNN). For every 400 training tasks, we tested the model for another 400 tasks sampled from the validation set (if available). If the model performance exceeded the previous best validation result, we applied it to the test set. Following the previous approaches compared in Table 1, we sampled another 400 tasks randomly drawn from the test set and report the average accuracy. Table 1 demonstrates that our adaCNN model achieves competitive results on the Omniglot tasks.
Overall we can observe a ceiling effect among the best performing models on these tasks. On the other hand, there exists room for improvement on the Mini-ImageNet task, as shown in Table  2. We obtained 57.1% and 70.04% accuracy on the Mini-ImageNet 5-way classification with the adaResNet model, slightly improving on the previous best result of the TCML model. Comparing different CNN architectures, indeed the more sophisticated adaResNet model yields almost 10% of improvement over the simpler adaCNN model on this task. The DF meta information performs competitive while being less expensive to obtain.

One-shot Language Modeling
We evaluated two main variants of our adaptive neuron models on 1-, 2-, and 3-shot language modelling (LM) tasks. For the first model we stack 3-layer feed-forward net with the adaptive neurons (adaFFN) on top of an LSTM network (LSTM+adaFFN). In this model, only the adaFFN gets to adapt to the task specific bias to perform the few-shot 5-way classification by taking in the hidden state of the underlying LSTM encoder. Therefore the LSTM encoder builds up the context for each sentence and provides generic representation to adaFFN. Both components are trained jointly.
The second model we propose for this problem is more flexible, which is LSTM with the adaptive neurons (adaLSTM). The entire model is adapted with task specific biases in every time step. For the few-shot classification output, a thin softmax layer which also has the adaptive neurons is stacked on top of adaLSTM. Comparing LSTM+adaFFN and adaLSTM, the former is much faster as we only adapt the activations of FFN three layers, yet lacks a full flexibility since the standard LSTM used is not aware of the current task information. We also evaluated deep (2-layer) versions of both LSTM+adaFFN and adaLSTM models. Once trained the baseline LSTM-LM models, we used it to score five candidate words and select the best scoring word for a test sentence. This is a common practice in machine reading comprehension tasks [11,10].
In terms of the evaluation protocol, we took two different ways to form test tasks. In the first approach, we randomly sample 400 hundred tasks from the test data and report the average. In the second approach, we make sure to include all the test words in task formulation. We randomly partition the 1000 target words into 200 groups and solve each group as a task. This approach is rather all-inclusive while in the conventional approach there is a chance that a word could be missed or included multiple times in different tasks. However, the conventional approach allows for formulation of an exponential number of test tasks. Table 3 summarizes our results. Our LSTM-LM baseline achieved 59.8/61.5% accuracy in 400random/all-inclusive settings respectively while upper bound provided by the oracle LSTM-LM is 72.8%. Our best accuracy is obtained by 2-layer adaLSTM which improves a simple Matching Nets' result by 24-32%, outperforming or competing with the well-tuned LSTM-LM baseline. Comparing the model variants, the adaLSTM model performs consistently well across the board and the deeper models yield a higher accuracy. Providing more sentences for the target word increases the performance. However we observed a saturation when the performance approaches 70% as the oracle score is 72.8%. With the all-inclusive evaluation, the models tend to get higher accuracy.
Overall the DF meta information slightly under-performs against the error gradient based one. We also observed difficulty in optimizing the LSTM+adaFFN models with the DF meta information (i.e. LSTM+adaFFN (DF)), causing no improvement in the training loss. When there is no improvement in the training loss in early stages of the training, we halted and restarted the run. We did not observe such difficulty in the adaCNN and adaResNet models when used the DF meta information.

Conclusion
We introduced a rapid-temporal adaptation (RTA) framework and a neuron level adaptation -adaptive neurons for artificial neural networks. The RTA framework builds upon an idea of meta information about a learning model. We proposed a novel direct feedback (DF) meta information that can be extracted for artificial neurons efficiently without having to use a sequential error backpropagation procedure. The adaptive neurons are not only generic enough to be incorporated in various deep neural network architectures, also computationally efficient than the methods that adapt synaptic connections between neurons. We improved previous results on standard benchmarks with the adaptive neurons and showed that the proposed DF meta information performs as competitively well as the error gradient based meta information. A potential future work would be set to see whether the adaptive neurons can be useful in other setups where training and inference of models involve conditioning on external incoming data.